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Abstract 

Th«  primary  purpose  of  this  investigation  was  to 
develop  an  acquisition  management  strategy  applicable  to  DOD 
program  management  that  would  help  program  managers  achieve 
acquisition  success.  An  hypothesized  management  strategy 
was  formulated  from  the  exploration  of  the  successful 
Federal  Aviation  Administration's  Host  computer  program. 

This  exploration  used  personal  interviews  to  identify  those 
management  elements  and  organizational  procedures  perceived 
by  the  28  respondents  to  contribute  to  Host  success.  The 
hypothesized  management  strategy  was  subsequently  evaluated 
by  experts  in  DOD  acquisition.  Through  personal  interviews 
each  element  in  the  management  strategy  was  evaluated  for 
necessity  to  achieve  program  success  and  applicability  to 
DOD  programs. 

The  conclusions  and  recommendations  of  the  study  were 
based  on  the  results  of  the  DOD  acquisition  expert  opinion 
survey  of  the  hypothesized  management  strategy.  The  result 
was  a  management  strategy  to  guide  DOD  program  managers  in 
achieving  acquisition  success  that  can  be  tailored  to  all 
programs . 
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ANALYSIS  OF  THE  FEDERAL  AVIATION  ADMINISTRATION’S 

HOST  COMPUTER  ACQUISITION  PROCESS  AND  POTENTIAL 
APPLICATION  IN  DEPARTMENT  OF  DEFENSE  ACQUISITIONS 

I .  Introduction 

General  Issue 

There  have  been  many  positive  achievements  in  the 
Defense  Department's  management  of  major  acquisitions  in  the 
past  several  years,  yet  an  obvious  need  exists  for 
additional  improvements.  Many  critics  feel  that  defense 
acquisition  organization  and  procedures  must  be  revitalized. 
In  July  1985,  in  response  to  this  perceived  need,  the 
President’s  Blue  Ribbon  Commission  on  Defense  Management  was 
charged  to  conduct  a  study  of  current  defense  management  and 
organization.  Acquisition  organization  and  procedures  were 
investigated  as  one  of  five  areas  of  concern. 

The  preliminary  objective  of  the  Commission  was  to 
identify  the  structural  problems  limiting  success  in  each  of 
the  five  areas.  As  pointed  out  in  An  Interim  Report  to  the 
President .  '...there  is  legitimate  cause  for  dissatisfaction 
with  the  process  by  which  the  Department  of  Defense  and 
Congress  buy  military  equipment  and  materiel'  (President’s 
Commission,  1986c:  15).  The  Commission  clarifies  the  reason 
for  dissatisfaction  with  the  following  statement:  ‘With 
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notable  exceptiona  iraapon  ayat«ma  taka  too  long  and  cost  too 
much  to  produce.  Too  often  they  do  not  perform  as  promised 
or  expected*  (President's  Commission,  1986c: 13).  These 
shortfalls  have  a  devastating  effect  on  military  readiness, 
and  they  preclude  the  attainment  of  maximum  benefit  from 
taxpayer’s  dollars. 

In  the  forward  to  A  Quest  for  Excellence.  Final  Report 
to  the  President.  Mr.  David  Packard,  the  Commission 
Chairman,  concludes  that,  'Excellence  in  defense  management 
will  not  and  can  not  emerge  by  legislation  or  directive. 
Excellence  requires  the  opposite  -  responsibility  and 
authority  placed  firmly  in  the  hands  of  those  at  the  working 
level,  who  have  knowledge  and  enthusiasm  for  the  tasks  at 
hand'  (President’s  Commission,  1986a: forward  xii) . 

Problem  Statement 

A  successful  acquisition  process  must  meet  budget, 
schedule  and  reliability  goals,  and  result  in  a  product  that 
meets  user  needs.  As  many  critics  of  the  current  DOD 
acquisition  process  point  out,  this  is  not  easily  done. 

The  responsibility  for  achieving  acquisition  success 
lies  with  the  program  manager  (PM) .  The  system  acquisition 
management  concept  designates  a  single  manager  to  be 
responsible  for  all  of  the  technical  and  business  aspects  of 
a  program.  The  designated  program  manager  is  given  the 
authority  required  to  meet  the  broad  and  multiple 
requirements  of  system  acquisition  and  is  held  accountable 
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for  the  accomplishment  of  the  total  management  task.  The 
program  manager  must  assemble  a  management  team  of  personnel 
from  all  functional  areas  to  assist  in  accomplishing  the 
program  objectives.  As  the  agent  of  the  service  in  the 
management  of  the  system  acquisition,  and  the  focal  point  of 
authority  and  responsibility  for  program  accomplishment,  the 
program  manager  has  a  demanding  and  complex  job. 

The  Federal  Acquisition  Regulations  and  DOD  Directives 
and  Instructions  concerning  acquisition  provide  guidance  for 
program  managers.  This  guidance  is  in  terms  of  end 
objectives  and  progress  milestones,  leaving  the  program 
manager  with  the  complex  task  of  determining  how  to  achieve 
those  objectives  and  milestones.  For  example,  DODD  5000.1, 
“Major  and  Mon-tlaJor  Defense  Acquisition  Programs.*  presents 
the  policies  of  the  Secretary  of  Defense  and  requires 
program  manager  compliance.  The  intent  is  to  provide 
guidance.  It  is  not  intended  to  foresee  all  problems, 
provide  all  solutions,  or  outline  all  task  details.  A  basic 
management  strategy  for  program  managers  that  provides  a 
flexible  framework  of  the  elements  and  procedures  key  to 
achieving  program  objectives  would  effectively  augment 
current  guidance. 

The  President’s  Blue  Ribbon  Commission  guidance  on  bow 


to  improve  the  acquisition  process  indicates  that  it  can 
best  be  improved  through  effective  management  procedures  and 
organization  elements.  In  An  Interim  Report  to  the 


Pr»«id»nt .  th«  Commission  statss.  'The  nation's  defense 
programs  lose  far  more  to  inefficient  procedures  than  to 
fraud  and  dishonesty'  (President's  Commission,  1986c:15}. 

Based  on  the  Commission  recomsiendations  and  the 
noticeable  lack  of  management  principles  in  current 
guidance,  program  managers  would  greatly  benefit  from  a 
management  strategy  composed  of  the  procedures  and 
organizational  elements  that  have  increased  the 
effectiveness  and  success  of  other  programs.  This  strategy 
would  simplify  the  program  management  task  by  providing  a 
proven  framework  to  start  with,  and  to  work  within.  The 
objective  of  this  research  is  to  use  recommendations  from 
current  studies,  lessons  learned  from  a  successful  program, 
and  DOD  expert  opinions  to  develop  an  acquisition  manageaient 
strategy  that  program  managers  can  tailor  to  their  specific 
program  to  achieve  program  success. 

Backn round 

The  Department  of  Defense  Acquisition  System  is  a 
single  uniform  system  of  policies  and  practices  that  governs 
how  equipment,  facilities,  and  services  are  planned, 
designed,  developed,  acquired,  maintained  and  disposed  of 
within  the  Department  of  Defense  (DOD,  1987:1).  DOD 
Directive  5000.1,  Major  and  Non-Major  Defense  Acquisition 
Programs  (DODD  5000.1),  and  DOD  Instruction  5000.2,  Defense 
Acquisition  Program  Procedures  (DODI  5000.2),  provide 
policies  and  procedures  for  managing  defense  acquisition 
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programa .  This  Dirsctlvs  and  corrasponding  Instruction 
take  precedence  over  all  other  DOD  issuances,  and  apply  to 
all  organizational  levels  from  the  Office  of  the  Secretary 
of  Defense  to  DOD  Field  Activities.  All  DOD  issuances 
providing  additional  guidance  for  managing  defense 
acquisition  programs  are  required  to  be  reviewed  for 
conformity  with  DODD  5000.1  and  DODI  5000.2.  If  an  issuance 
is  found  to  be  in  conflict,  it  must  be  changed  or  cancelled 
as  appropriate. 

DODD  5000.1  presents  the  policies  of  the  Secretary  of 
Defense  to  govern  acquisition  programs  and  requires  program 
manager  compliance.  The  policies  provided  cover  many 
issues.  Those  that  apply  to  program  management  are: 

1.  Support  national  policy  and  operational  objectives 
with  a  timely,  efficient  and  effective  acquisition  system. 

2.  Streamline  the  acquisition  organization  structure. 

3.  Establish  acquisition  phases  and  milestone  decision 
points  and  to  enhance  management  effectiveness. 

4.  Enhance  program  stability  by  minimizing  program 
changes,  by  providing  realistic  estioiates,  and  by 
establishing  program  baselines. 

5.  Tailor  the  acquisition  strategy  to  meet  the  unique 
circuBistances  of  Individual  programs.  (DOD,  1987a:  3-6) 

The  DOD  acquisition  life  cycle  is  divided  into  five 
phases  which  are  separated  by  decision  milestones.  DOD 
Directive  5000.1  policy  requires  the  use  of  acquisition 
phases  and  milestone  decision  points  to  manage  major  defense 
acquisition  programs  and,  where  applicable,  non-major 
acquisition  programs.  It  is  important  to  note  that  the 
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Dir«ctiv«  specifically  states ,  'These  phases  are  to  be 
tailored  to  fit  each  acquisition  to  minimize  acquisition 
time  and  life-cycle  costs...*  (DOD.  1987a:3).  If 
appropriate,  phases  may  be  omitted  or  combineo. 

The  decision  milestones  are  designed  to  develop  a  level 
of  confidence  in  the  solution  offered  and  to  reduce  the 
degree  of  technical  risk  involved.  At  the  end  of  each 
phase,  validated  test  results  form  the  information  base  for 
the  decision  to  proceed,  to  redirect  and  study  further,  or 
to  terminate  programs.  The  incremental  commitment  of 
resources  with  each  decision  to  proceed  also  reduces  risk 
(McCarty,  1987:5). 

OOD  Instruction  5000.2  provides  the  decision  criteria 
and  considerations  for  each  milestone.  Action  to  initiate 
each  phase  is  contingent  upon  the  decision  reached  at  the 
preceding  milestone.  The  acquisition  phases  and  milestone 
decisions  of  the  system  acquisition  life  cycle  are  outlined 
below  in  chronological  order  (DOD,  1987b: 2-4;  Long  and 
others,  1987:149-159;  McCarty,  1987:8-20): 

1.  In  the  operational  requirements  phase  operational 
deficiencies  are  identified,  operational  needs  are  stated, 
and  the  developsient  or  improvement  of  systems  or  equipment 
is  initiated. 

2.  The  Milestone  0  decision  determines  mission-need 
and  approves  program  initiation  and  authority  to  budget  for 
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«  new  program.  The  commitment  extends  only  to  identifying 
and  exploring  alternative  solutions. 

3.  In  the  concept  exploration/definition  phase 
alternative  solutions  defined  at  Milestone  0  are  evaluated, 
and  the  best  is  selected.  Limited  experiments  and  tests  may 
be  conducted.  Some  of  the  primary  considerations  are:  1) 
mission  area  analysis;  2)  affordability  and  life  cycle 
costs;  and  3)  operational  utility  assessment. 

4.  The  Milestone  I  decision  considers  proceeding  with 
the  concept  demonstration/validation  phase.  Broad  program 
cost,  schedule,  and  operational  effectiveness/suitability 
goals  and  thresholds  are  established  for  the  alternative (s) 
selected  in  the  preceding  phase. 

5.  In  the  concept  demonstration/validation  phase  the 
technical  risk  and  economic  xjuncertainty  are  reduced  through 
a  more  detailed  definition  of  the  new  system.  Primary 
system  hardware  prototyping,  studies  and  analysis,  or 
subsystem  prototyping  with  system  analysis  are  used  to 
further  define  the  system. 

6.  The  Milestone  II  decision  considers  proceeding  with 
the  full-scale  development  phase,  and  establishes  more 
specific  goals  and  thresholds.  An  affirmative  decision  at 
Milestone  II  is  a  commitment  to  continue  the  program  through 
engineering  development  and  to  acquire  long-lead  procurement 
items  required  to  support  operational  testing  and  initial 
production . 
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7.  In  the  full-scale  development/low  rate  initial 
production  phase  the  system  design  is  revieived  and 
finalized,  and  the  system  is  developed,  produced  and  tested. 
Test  and  evaluation  is  performed  to  minimize  design  risk,  to 
demonstrate  that  the  system  meets  performance 
specifications,  and  to  demonstrate  achievement  of  program 
objectives.  The  output  from  this  phase  includes  a  pre- 
production  prototype,  results  of  completed  operational  test 
and  evaluation,  production  or  construction  cost 
verification,  the  production  and  deployment  schedule, 
affordability  and  life  cycle  coats,  plana  for  integrated 
logistics  support,  and  a  procurement  strategy. 

8.  The  Milestone  III  decision  considers  proceeding 
with  the  full-rate  production/deployment  phase.  An 
affirmative  decision  here  defines  the  initial  production 
quantity  and  approves  plans  for  future  production. 

9.  In  the  full-rate  production/deployment  phase,  the 
actual  production  commitment  is  formally  and  contractually 
accomplished.  Testing  begun  in  the  full-scale  development 
phase  is  continued  until  system  perforoiance  specification 
requirements  are  met.  The  system,  which  includes  training 
equipment,  spares,  facilities  and  support  equipment,  is 
produced  for  operational  use.  Deployment  begins  when  a 
system  in  operational  configuration  is  provided  to  and  used 
by  operational  units. 
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10.  The  operational  support  phase  begins  at  the  point 
of  initial  operational  capability,  when  the  first  item  is 
deployed . 

11.  The  Milestone  IV  decision  identifies  actions  and 
resources  needed  to  ensure  that  operational  readiness  and 
support  objectives  are  achieved  and  maintained  for  the  first 
several  years  of  the  operational  support  phase.  This 
milestone  normally  occurs  one  to  two  years  after  initial 
deployment . 

12.  The  Milestone  V  decision  encompasses  a  review  of  a 
system’s  current  state  of  operational  effectiveness, 
suitability,  and  readiness  in  order  to  determine  whether 
major  upgrades  are  necessary  or  deficiencies  warrant 
consideration  of  replacement.  This  decision  point  normally 
occurs  five  to  ten  years  after  initial  deployment. 

Def initions 

The  following  terms  are  defined  in  order  to  clarify  the 
purpose,  analysis,  and  recommendations  of  this  study. 

1.  Criteria  for  Success  ~  The  criteria  that  must  be 
satisfied  for  an  acquisition  to  be  considered  successful  for 
the  purpose  and  scope  of  this  study.  An  acquisition  is 
considered  successful  if:  costs  do  not  exceed  budget;  the 
original  schedule,  or  date  of  delivery,  is  met;  the  product 
meets  reliability,  maintainability,  and  availability 
standards;  and  the  product  is  accepted  by  the  user  and  meets 
the  user’s  needs. 
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2.  Elements  for  Success 


the  acquisition  management 


procedures  and  organizational  elements  determined  by 
previous  research  and  by  analysis  conducted  in  this  thesis 
to  increase  the  potential  for  successful  acquisition. 
Abbreviated  as  ‘elements.* 

Research  Objectives 

The  objectives  of  this  research  were: 

1.  To  discover  and  analyze  the  elements  for  success 
used  to  manage  a  successful  acquisition  program. 

2.  To  develop  an  acquisition  management  strategy  based 
on  the  elements  for  success  identified  that  is  applicable  to 
DOD  program  management. 


Investisati ve  Questions 

This  research  effort  answered  the  following 
investigative  questions: 

1.  What  criteria  must  be  met  for  an  acquisition 
program  to  be  considered  successful? 

2.  What  management  procedures  and  organizational 
elements  were  recommended  in  previous  research  to  improve 
the  DOD  acquisition  process? 

3.  What  are  the  manageoient  procedures  and 
organizational  elements  used  by  the  program  management  team 
of  a  successful  development  type  acquisition? 

a.  Does  the  program  meet  the  criteria  established 
in  the  literature  review  for  a  successful  acquisition? 

b.  Of  the  elements  used,  which  are  perceived  by 
the  progra  <1  management  team  to  be  vital  to  the  success  of 
their  program? 

4.  How  do  the  management  procedures  and  organizational 
elements  recommended  in  previous  research  compare  to  those 
perceived  by  the  program  management  team  to  be  vital  to  the 
success  of  their  program? 


5.  Which  of  the  elementa  for  success  resulting  from 
the  analysis  of  the  successful  program  and  previous  research 
form  the  management  strategy  that  will  potentially  increase 
the  DOD  acquisition  success  rate? 

a.  Which  of  these  elements  for  success  do  DOD 
acquisition  experts  identify  as  ideas  new  to  DOD? 

b.  Which  of  these  eleskenta  for  success  do  DOD 
acquisition  experts  identify  as  necessary  for  program 
success? 


c.  Of  the  elements  identified  by  DOD  acquisition 
experts  as  necessary  for  program  success,  which  are 
currently  used  and  which  are  potentially  applicable  to  the 
DOD  acquisition  process? 

d.  What  must  be  done  before  the  elements  for 
success  potentially  applicable  to  the  DOD  acquisition 
process  can  be  implemented? 

e.  For  those  elements  identified  by  DOD 
acquisition  experts  as  not  potentially  applicable  to  the  DOD 
acquisition  process,  why  were  they  so  identified? 

f.  For  those  elements  identified  by  DOD 
acquisition  experts  as  currently  used  or  potentially 
applicable,  what  is  the  range  of  applicability? 


Scope 

This  research  effort  focused  on  the  oianagement  strategy 
of  the  program  manager  from  contract  development  to  system 
acceptance.  This  is  the  time  period  when,  according  to 
current  regulations  and  guidance,  program  performance  is  up 
to  the  management  and  leadership  abilities  of  the  PM. 
Therefore,  this  is  the  time  period  that  an  acquisition 
management  strategy  is  most  appropriate. 

It  was  required  that  the  successful  acquisition  program 
selected  for  the  exploratory  study  be  sponsored  by  a 
government  agency  or  industry  other  than  the  DOD.  The 
objective  behind  this  requirement  was  to  identify  new  ways 
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of  handling  and  managing  the  section  of  the  acquisition  life 
cycle  being  reviewed  by  this  research  effort.  The  program 
selected  for  the  exploratory  study  was  required  to  be 
complete  or  near  completion.  This  was  necessary  so  that  all 
relevant  phases  of  the  acquisition  life  cycle  could  be 
evaluated,  and  so  that  program  success  could  be  determined. 

Overview 

The  next  five  chapters  include:  a  Literature  Review  of 
contemporary  reports  and  recommendations  on  improving 
acquisition  procedures  and  organizational  elements  (Chapter 
ID ;  the  Research  Methodology  (Chapter  III)  ;  the  results  of 
an  Exploratory  Study  of  the  Federal  Aviation  Administration 
Host  Computer  System,  a  Joint  analysis  of  the  elements 
identified  with  the  recommendations  from  the  Literature 
Review,  and  a  management  strategy  proposed  for  use  in  OOD 
acquisitions  (Chapter  IV) ;  the  results  of  an  Expert 
Interview  analysis  of  the  proposed  management  strategy 
(Chapter  V) ;  and  Conclusions  and  Recommendations 
(Chapter  VI) . 
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I I .  Literature  Review 


Chapter  Overview 

In  the  past  eeveral  years  defense  management  and 
defense  acquisition  procedures  have  been  the  subjects  of 
numerous  studies.  This  chapter  presents  a  review  of 
literature  related  to  the  findings  of  defense  acquisition 
management  studies.  The  review  focused  on  studies  conducted 
from  1984  through  1987.  The  initial  part  of  the  chapter 
presents  the  criteria  that  must  be  met  for  an  acquisition  to 
be  considered  successful  according  to  contemporary 
literature.  The  remainder  of  the  chapter  discusses  the 
program  management  procedures  and  organizational  elements 
recommended  to  improve  the  acquisition  program  success  rate. 
SoBie  of  problems  that  the  recommendations  were  proposed  to 
counter  are  also  presented. 

Two  major  studies  served  as  the  basis  for  revitalizing 
the  current  DOD  acquisition  system.  The  first  study,  by  the 
Defense  Science  Board,  Practical  Functional  Perfornaance 
Requirements .  compared  five  successful  commercial  programs 
with  28  defense  programs  and  served  as  the  basis  for  the 
President's  Blue  Ribbon  Commission  Study.  The  second  study, 
conducted  by  the  President’s  Blue  Ribbon  Commission  on 
Defense  Management,  compared  the  defense  acquisition  system 
with  other  systems,  both  government  and  commercial.  These 


two  studies  were  the  catalyst  for  the  September  1987  update 
of  DOD  Directive  5000.1  and  DOD  Instruction  5000.2.  They 
also  form  the  basis  of  this  literature  review. 

Criteria  Defining  Acquisition  Success 

To  answer  investigative  question  1  various  system 
acquisition  documents  were  reviewed.  It  was  found  that  the 
criteria  that  must  be  met  for  an  acquisition  to  be 
considered  successful  were  the  same  for  both  DOD  and  non-DOD 
programs.  According  to  the  Defense  Science  Board  study  of 
the  performance  requirements  aspect  of  the  acquisition 
process,  a  program  must  meet  the  following  criteria  to  be 
considered  successful  (Defense  Science  Board,  1986:18): 

1 .  The  equipment  is  placed  in  the  hands  of  the 
operating  forces  in  sufficient  quantities  to  assist  in 
deterring  a  war. 

2.  The  program  meets  the  projected  schedule.  The 
equipment  is  available  to  the  operating  forces  on  the  date 
projected. 

3.  The  equipment  does  not  exceed  the  projected  budget, 
or  cut  quantities. 

4.  The  equipment  performs  as  projected. 

The  Office  of  Management  and  Budget  (0MB)  outlined  the 
policies  to  be  followed  by  the  executive  branch  agencies  in 
the  acquisition  of  major  systems  in  0MB  Circular  A-109. 
According  to  this  circular  a  successful  program  must  meet 
program  objectives.  The  program  objectives  were  defined  as 
the  capability,  cost,  and  schedule  goals  being  sought  by  the 
system  acquisition  program  (0MB,  1976:2).  In  addition  to 
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these  program  objectives  General  Bernard  Randolph,  Commander 
of  the  Air  Force  Systems  Command,  specified  that  all 
Individuals  in  AFSC  would  be  measured  by  how  they  supported 
the  using  Operational  Commanders  (Department  of  the  Air 
Force,  1987c).  This  measurement  criterion  was  extended  to 
measuring  acquisition  success. 

The  criteria  outlined  in  system  acquisition 
documentation  were  evaluated  in  conjunction  with  the 
criteria  specified  by  several  program  management  experts. 

It  was  found  that  the  criteria  for  determining  acquisition 
success  were  conceptually  alike.  The  various  criteria 
collected  differed  only  in  format.  Based  on  these  sources, 
the  following  list  of  criteria  that  must  be  met  for  an 
acquisition  to  bo  considered  successful  answer  investigative 
question  1: 

1.  Costs  do  not  exceed  budget. 

2.  Original  schedule,  or  date  of  delivery,  met. 

3.  Product  meets  projected  performance  standards. 

4.  The  product  is  accepted  by  the  user,  and  meets  the 
user’s  needs. 

Recommendations  For  a  Successful  Program 

This  portion  of  the  chapter  presents  the  management 
procedures  and  organizational  elements  recommended  in 
previous  research  to  improve  the  DOD  acquisition  process. 
This  information  is  used  to  answer  investigative  question  2. 
Only  those  recommendations  within  the  scope  of  management 
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actions  that  can  be  implemented  by  the  program  manager  to 
improve  the  program  success  rate  will  be  addressed.  The 
following  paragraphs  briefly  describe  the  studies  evaluated 
in  this  literature  review. 

The  recommendations  in  the  Defense  Science  Board  report 
were  aimed  at  strengthening  the  process  by  which 
requirements  are  generated,  iterated,  and  implemented 
(Defense  Science  Board,  1986:  Forward,  v) .  This  board 
believed  that  despite  differences  in  the  development 
processes  of  commercial  and  military  programs  there  were 
some  major  lessons  learned  that  could  be  Judiciously  applied 
to  military  programs  (Defense  Science  Board,  1986;  Forward, 

V)  . 

The  President  established  the  Blue  Ribbon  Commission  on 
Defense  Management  to  evaluate  the  defense  acquisition 
system,  to  determine  how  it  might  be  improved,  and  to 
recommend  changes  that  could  increase  the  rate  of  program 
success  (President’s  Commission,  1986b: 1).  The  Commission 
examined  both  DOD  and  non-DOD  programs  that  had  been  most 
successful  in  acquisition  in  order  to  find  a  model  of 
excellence  for  defense  acquisition.  They  concluded  that 
meaningful  improvement  would  not  come  from  more  regulation, 
but  from  major  institutional  change.  The  recommendations 
advocated  a  new  theory  of  management  similar  to  that  applied 
in  Japanese  companies  and  some  U.S.  companies  credited  with 
increasing  productivity  and  quality.  Through  their 
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reconusendationa  tha  Comniaaion  hoped  to  inatill  the 
following  *new‘  management  theory  Into  DOD  acqulaitlon: 

1)  participation  of  all  of  the  people  involved  with  the 
program,  2)  truat  in  people  and  belief  that  people  in  the 
organization  want  to  do  a  good  job,  3}  people  given  the 
opportunity  to  contribute  their  knowledge,  akill,  and 
enthuaiaam  to  work  together  to  achieve  the  goala  of  the 
organization,  and  4)  auperviaion  minimized,  and  detailed 
review  of  work  reduced  (Preaident'a  Commiaaion,  1986a: 

41-42)  . 

The  Defenae  Organization  Project  of  Georgetown 
Univeraity’a  Center  for  Strategic  and  International  Studiea 
waa  initiated  to  identify  thoae  actiona  that  tvould  help  to 
build  a  more  effective  and  reaponaive  defenae  atructure 
(Ganaler,  1985:68-90).  Three  profeaaora  of  Syatema 
Acqulaitlon  Management  for  the  Department  of  Reaearch  and 
Information  at  the  Defenae  Syatema  Management  College,  Ft 
Belvoir  VA,  conducted  a  atudy  to  find  out  *what  makea  for 
aucceaa  in  aystema  acqulaitlon  management*  and  to  identify 
*what  we  have  been  doing  right  in  defenae  ayatema  management 
that  we  want  to  repeat*  (Baumgartner,  1984:31).  They 
aelected  12  programa  from  the  52  identified  by  the  Joint 
Logiatica  Commandera  aa  aucceaaful.  From  thoae  12  programa 
47  program  managera  and  their  induatry  counterparta  were 
interviewed . 
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Th«  following  sections  present  the  recommendations  of 
these  studies  that  apply  to  program  management  and  can  be 
Implemented  by  program  managers.  A  section  Is  devoted  to 
each  recommendation.  These  sections  cumulatively  provide 
the  answer  to  Investigative  question  2. 

Program  Manager  Authority  and  Flexibility.  A  primary 
recommendation  of  all  of  the  studies  was  for  program 
managers  to  be  given  the  authority  to  stake  trade>off 
decisions  on  their  programs,  the  authority  to  reject  changes 
to  the  program  baseline,  and  the  authority  to  execute  the 
program  (Defense  Science  Board.  1080:2;  Oansler,  1985:03; 
President’s  Commission,  1980b ; 12 , 22) .  This  recommendation 
Is  not  a  management  procedure  to  be  employed  by  the  program 
Bianager.  It  Is  Included  because  siany  of  the  recosimendatlons 
listed  In  subsequent  sections  of  this  chapter  could  not  be 
Isiplemented  unless  the  program  manager  had  sufficient 
authority  to  accomplish  them. 

Stabilize  ProArams .  The  need  for  program  and  budget 
stability  was  recognized  by  the  President’s  Blue  Ribbon 
Commission,  and  by  the  Defense  Organization  Project  Working 
Qroup  on  Weapons  Acquisition  (Qansler,  1985:93-04; 
President’s  Commission,  1986a: 59-60) .  Program  stability  was 
seen  as  necessary  to  counter  growth  In  the  cost  of  defense 
equipment  due  to  program  stretch-outs.  Both  groups 
recommend  that  program  managers  make  realistic  program  cost 


estimates.  Including  factors  to  cover  the  risks  Inherent  in 


the  development  of  weapon  ppograms .  The  main  point 
emphasized  by  the  Weapons  Acquisition  Working  Group  was  that 
even  a  well-planned  and  well-run  program  could  not  succeed 
unless  the  dollars  were  available  to  fund  it.  In  addition 
to  budget  stability,  both  groups  recommended  that  more 
research  be  conducted  at  the  outset  of  weapons  development 
to  coxjnter  overstated  specifications,  and  that  program 
managers  be  given  the  authority  to  reject  changes  suggested 
by  other  staffs.  They  emphasized  that  once  the  baseline  was 
established  the  program  manager  must  strictly  adhere  to  it. 

A  baseline  was  defined  as  the  agreement  drafted  by  the 
program  manager  describing  functional  specifications,  cost, 
schedule,  and  other  factors  critical  to  the  program’s 
success  submitted  for  approval  to  the  Defense  Acquisition 
Executive  to  which  the  program  manager  would  be  held 
accountable  (President’s  Commission,  I986a:59). 

Small.  High  Quality  Staffs.  The  authority  to  hand- 
select  high  quality  personnel  was  one  of  the  features  of 
most  successful  commercial  programs  identified  by  the 
President’s  Blue  Ribbon  Commission  (President’s  Commission, 
1986b: 12 , 13 , 27) .  In  view  of  this  element  for  success,  the 
Commission  made  several  recommendations.  The  recommendation 
moat  applicable  to  the  scope  of  this  study  was  to  establish 
flexible  personnel  management  policies  necessary  to  improve 
defense  acquisition. 
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Proilram  Offic*  T«am  AinwaphT*.  A  recoinm«nd*tion 
r««ultlnj|  from  the  Defans*  Systems  Management  College 
research  effort  was  for  the  program  sianager  to  delegate 
authority  and  responsibility,  and  to  create  a  team 
atmosphere  (Baumgartner,  1984:30-36).  The  program  managers 
interviewed  believed  that  the  recommended  actions  would  help 
to  train  people,  increase  retention  of  good  people,  and  free 
the  program  manager  to  handle  the  big  problems.  The  need  to 
hold  Individuals  accountable,  and  to  involve  everyone  was 
also  emphasized. 

CoBimuni  cat  ions  With  Users.  The  need  to  establish  a 
dialogue  with  the  customer,  or  user,  at  the  conception  of 
the  program  and  maintain  that  communication  throughout  the 
program  was  one  of  the  features  of  most  successful  programs 
identified  by  the  President’s  Blue  Ribbon  Coouniss ion 
(President's  Commission,  1980b : 12 , 13) .  It  was  noted  that  a 
number  of  successful  OOD  prograois  had  incorporated  this 
management  feature.  Communication  with  users  was  also  seen 
to  result  in  motivating  the  program  manager  to  seek  out  and 
address  problems,  rather  than  hide  them. 

The  Defense  Science  Board  recommended  that  the  role  of 
the  user  in  the  overall  DOD  guidance  and  requirements 
process  be  expanded  (Defense  Science  Board,  1980:4,  45-47, 
52).  The  board  found  that  the  user's  role  and 
responsibilities  were  vital  to  the  development  of 
performance  requirements.  This  recommendation  included  the 
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rvquireiMnt  for  program  managers  to  ensure  that  users  were 
cognizant  of  program  and  budget  priorities  and  tradeoffs. 

It  was  also  identified  that  the  role  of  users  should  be  made 
more  formal  early  in  the  requirements  generation  cycle,  and 
that  feedback  should  be  provided  on  a  continuing  bases. 

Prototyping  and  Testing.  The  need  to  establish  a 
devil’s  advocate  within  the  program  office  to  seek  out 
operational  pitfalls,  the  need  for  prototyping,  and  the  need 
for  early  operational  testing  were  features  of  most 
successful  programs  identified  by  the  President’s  Blue 
Ribbon  Commission  (President’s  Commission,  1986b: 
12,13,18-20).  These  needs  were  reflected  in  the 
recommendation  that  operational  testing  should  begin  early 
in  advanced  development  using  prototype  hardware.  In 
describing  this  recommendation  the  Commission  specified  that 
operational  testing  should  be  combined  with  developmental 
tests  of  the  prototype  to  uncover  operational  as  well  as 
technical  deficiencies.  It  was  noted  that  this 
recommendation  for  prototyping  and  testing  would  also 
contribute  materially  to  Improving  cost  and  schedule 
estimates . 

Adherence  to  Schedule.  The  Defense  Science  Board 
recommended  that  program  managers  strictly  adhere  to  the 
established  schedule  following  acquisition  life  cycle 
Milestone  II  (Defense  Science  Board,  1986:74,75).  Schedule 
dominance  was  viewed  as  a  means  of  preserving  program 


stability  and  constraining  cost  growth.  This  recommendation 
also  implied  that  requirements  were  validated,  and  that 
there  was  fvinding  stability. 

Teamwork  Relationship  Between  Government  and 
Contractor .  The  recommendation  ranked  most  important  by  the 
program  managers  interviewed  for  the  Defense  Systems 
Management  College  research  effort  was  to  establish  a 
teamwork  relationship  of  mutual  trust  betwreen  the  government 
and  contractor  program  management  (Baumgartner,  1984:36-38). 
The  need  for  open  communication,  a  good  interface,  and  joint 
discussion  of  problems  were  identified  to  be  vital. 

Conclusion 

This  summary  of  the  literature  reviewed  provides  the 
answers  to  investigative  questions  1  and  2.  The  next 
chapter  presents  the  detailed  methodology  employed  in 
answering  the  remaining  investigative  questions. 
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III.  Mathodolo 


ChaptT  Overview 

This  chapter  describes  the  research  methods  and 
specific  steps  used  to  answer  the  investigative  questions 
supporting  the  two  research  objectives.  For  the  reader’s 
convenience,  the  research  objectives  and  supporting 
investigative  questions  outlined  in  Chapter  I  are  restated 
here.  Second,  the  general  approach  of  the  research  is 
presented.  Next,  the  investigative  method  and  analysis 
process  used  to  meet  the  first  research  objective  are 
identified  and  described,  followed  by  those  for  the  second. 
The  chapter  concludes  with  a  summary  of  the  research 
approach . 

Research  Objectives  and  Supporting  Investisative  Questions 

Research  Objective  I.  To  discover  and  analyze  the 
elements  for  success  used  to  manage  a  successful  acquisition 
program.  Supported  by  the  following  investigative 
questions : 

1.  What  criteria  must  be  met  for  an  acquisition 
program  to  be  considered  successful? 

2.  What  management  procedures  and  organizational 
elements  were  recommended  in  previous  research  to  improve 
the  DOD  acquisition  process? 

3.  What  are  the  management  procedures  and 
organizational  elements  used  by  the  program  management  team 
of  a  successful  development  type  acquisition? 
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a.  Doas  the  program  meet  the  criteria  established 
in  the  literature  review  for  a  successful  acquisition? 

b.  Of  the  elements  used,  which  are  perceived  by 
the  program  management  team  to  be  vital  to  the  success  of 
their  program? 

4.  How  do  the  management  procedures  and  organizational 
elements  recommended  in  previous  research  compare  to  those 
perceived  by  the  program  management  team  to  be  vital  to  the 
success  of  their  program? 

Besearch  Objective  11.  To  develop  an  acquisition 
management  strategy  baaed  on  the  elements  for  success 
identified  that  is  applicable  to  DOD  program  otanagement. 
Supported  by  the  following  investigative  questions: 

5.  Which  of  the  elements  for  success  resulting  from 
the  analysis  of  the  successful  program  and  previous  research 
form  the  management  strategy  that  will  potentially  increase 
the  DOO  acquisition  success  rate? 

a.  Which  of  these  elements  for  success  do  DOD 
acquisition  experts  identify  as  ideas  new  to  DOD? 

b.  Which  of  these  elements  for  success  do  DOD 
acquisition  experts  identify  as  necessary  for  program 
success? 

c.  Of  the  elements  identified  by  DOD  acquisition 
experts  as  necessary  for  program  success,  which  are 
currently  used  and  which  are  potentially  applicable  to  the 
DOD  acquisition  process? 

d.  What  must  be  done  before  the  elements  for 
success  potentially  applicable  to  the  DOD  acquisition 
process  can  be  implemented? 

e.  For  those  elements  identified  by  DOD 
acquisition  experts  as  not  potentially  applicable  to  the  DOD 
acquisition  process,  why  were  they  so  identified? 


f.  For  those  elements  identified  by  DOD 
acquisition  experts  as  currently  used  or  potentially 
applicable,  what  is  the  range  of  applicability? 


G«nTal  Approach 

The  primary  assertion  of  thie  thesis  is  that  the 
exploration  of  a  successful  program  will  reveal  management 
processes  that  facilitate  achieving  program  success  that 
were  not  previously  recommended  in  research  nor  used  in  DOD 
program  management.  The  goal  is  to  use  recommendations  from 
current  studies,  lessons  learned  from  a  successful  program, 
and  DOD  expert  opinions  to  develop  an  acquisition  management 
strategy  that  DOD  program  managers  can  tailor  to  their 
specific  programs  to  achieve  program  success. 

The  general  approach  used  to  determine  the  validity  of 
the  primary  assertion  began  with  a  review  of  contemporary 
research  efforts  in  the  area  of  DOD  acquisition  management. 
This  review  resulted  in  the  definition  of  criteria  that  must 
be  met  for  a  program  to  be  considered  successful,  and  the 
identification  of  the  recommendations  for  improving  the  DOD 
acquisition  process.  Next,  a  successful  program  that  met 
all  of  the  criteria  of  success  was  selected  for  exploration. 
The  Federal  Aviation  Administration’s  Host  Computer  System 
was  the  program  selected.  Representatives  from  each 
component  of  the  Host  program  management  team  were 
interviewed  to  identify  which  managerial  procedures  and 
organizational  elements  they  perceived  to  contribute  to  Host 
success.  After  comparing  these  elements  with  the 
recommendations  of  current  studies,  a  management  strategy 


for  achieving  DOD  program  success  was  hypothesized.  In  the 


final  st«p.  the  opinions  of  OOD  acquisition  experts  were 
used  to  evaluate  the  hypothesized  management  strategy.  The 
result  was  a  managestent  strategy,  based  on  elements  actually 
used  in  a  successful  program,  that  was  applicable  to  the  DOD 
acquisition  process. 

Specific  Procedures  for  Answering  Besearch  Objective  I 

Discovering  and  analyzing  the  elements  for  success  used 
to  manage  a  successful  acquisition  program  implies  the  need 
to  Identify  the  criteria  by  which  a  program  is  Judged  to  be 
successful  or  not.  In  addition,  the  primary  assertion  of 
this  thesis  must  be  tested.  Investigative  questions  1  and  2 
addressed  these  needs,  and  were  answered  by  the  review  of 
contemporary  research  recommendations  for  improving  the  DOD 
acquisition  process.  The  results  of  the  review,  conducted 
in  response  to  these  questions,  are  reported  in  Chapter  II. 

The  response  to  the  first  investigative  question 
provided  the  foundation  necessary  to  identify  a  successful 
program.  In  August  1987,  two  Host  computer  system  experts 
were  interviewed  to  determine  the  scope  of  the  proposed 
research  area,  and  to  gather  background  data  on  the  Host 
system  acquisition.  In  the  second  week  of  September  1987, 
two  briefings  were  observed  and  several  exploratory 
interviews  were  conducted  at  the  Indianapolis  Air  Route 
Traffic  Control  Center  (ARTCC) .  Sufficient  background  data 
was  gathered  to  determine  that  Host  did  meet  all  of  the 
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criteria  of  success,  and  to  establish  an  appropriate 
research  methodology. 

The  next  step  In  satisfying  the  first  research 
objective  was  to  Identify  the  management  procedures  and 
organizational  elements  perceived  by  the  Host  program 
management  team  to  contribute  to  Host  success. 

Investigative  question  3  addressed  this  need,  and  was 
supported  by  an  exploratory  case  study  research  method. 
Personal  Intervl  ^  were  the  primary  source  for  the  data 
used  to  answer  question  3.  Selected  members  of  the  program 
management  team  were  Interviewed  at  FAA  Headquarters  In 
Washington  D.C.  on  25,  28  September,  and  on  22,  23  December 
1987.  During  these  visits,  additional  documentation  was 
gathered  as  necessary,  and  several  program  meetings  were 
observed.  The  documentation  and  observations  contributed  to 
content  validity,  and  were  used  to  corroborate  the  personal 
Interview  data. 

The  final  step  necessary  to  satisfy  the  first  research 
objective  was  analysis  of  the  data  collected.  This  need  was 
met  by  Investigative  question  4,  and  facilitated  by  the 
results  from  the  literature  review. 

The  following  sections  address  the  specific  elements 
that  compose  the  methodology  outlined  above.  First,  the 
literature  review  process  Is  presented,  followed  by  a 
discussion  of  why  the  Host  computer  system  was  selected  for 
evaluation.  Third,  the  exploratory  case  study  research 
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method  Is  defined,  subsequent  subsections  outline  the  data 
collection  plan,  and  discuss  the  interview  sample  selection, 
the  interview  process  protocol,  the  observation  sample 
selection,  and  the  observation  protocol.  Finally,  the 
method  of  data  analysis  is  presented. 

Literature  Review.  The  main  source  for  literature 
review  mater ials  was  Lt  Col  Delaney,  Chief,  Defense 
Resources  Division,  System  Acquisition  Management 
Department,  Air  Force  Institute  of  Technology,  School  of 
Systems  and  Logistics  at  Vrlght-Patterson  Air  Force  Base. 
Colonel  Delaney  identified  the  primary  reports  documenting 
the  recommendations  of  contemporary  research  conducted  in 
the  area  of  DOD  acquisition  management.  The  information  In 
these  sources  was  vital  to  this  research  effort. 

The  materials  available  in  the  AFIT  School  of  Systems 
and  Logistics  library  were  reviewed.  In  addition,  several 
topical  searches  were  conducted  through  the  Defense 
Technical  Information  Center.  These  efforts  provided  no 
significant  material  beyond  that  previously  identified. 

Successful  Acquisition  Program  Selection.  The  two 
primary  criteria  for  program  selection  were:  1)  all  criteria 
of  acquisition  success  achieved,  and  2)  key  personnel  and 
meetings  accessible.  The  Federal  Aviation  Administration 
Host  Computer  System  met  both  criteria. 

James  G.  Cain,  Deputy  Director  of  the  FAA  Advanced 
Automation  Program  Office,  identified  the  Host  Computer 
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System  as  one  o£  the  most  successful  technical  efforts  In 
PAA  history  (Office  of  Public  Affairs,  1987d:2).  The 
criteria  of  success  identified  in  the  literature  review,  and 
the  corresponding  evidence  of  Host  success  are  both  outlined 
below  (Garwood,  1987a;  Marek,  1987): 

1.  Criterion:  Costs  do  not  exceed  budget.  Host:  Host 
came  in  under  budget. 

2.  Criterion:  Original  schedule,  or  date  of  delivery, 
is  met.  Host:  Host  had  met  the  implementation  schedule  in 
all  areas. 

3.  Criterion:  The  product  meets  reliability, 
maintainability,  and  availability  (RMA)  standards.  Host: 

All  RMA  and  technical  performance  measurements  were  met  or 
exceeded.  This  was  documented  in  test  results. 

4.  Criterion:  The  product  is  accepted  by  the  user  and 
meets  the  user's  needs.  Host:  Every  control  center  formally 
accepted  the  system.  One  half  of  the  sites  requested 
permission  to  drop  some  preliminary  tests  and  accept  early. 
Every  control  center  continued  to  use  the  Host  system  after 
acceptance.  These  examples  of  user  confidence  are  evidence 
of  system  acceptance.  In  the  past  some  systems  have  been 
sent  back.  In  addition,  there  are  currently  some  devices  on 
location  that  site  managers  refuse  to  use. 

Accessibility  of  the  Host  program  was  ideal.  Two  Air 
Route  Traffic  Control  Centers,  key  locations  for  observing  a 
site  survey  and  a  dedication  ceremony,  were  within  200  miles 
of  Wrlght-Patterson  AFB  (W-PAFB).  The  FAA  Headquarters  in 
Washington  D.C.,  location  of  the  Host  program  management 
team,  was  a  short  distance  from  Andrews  AFB  which  was  served 
by  daily  operational  airlift  support  from  W-PAFB.  Most 
importantly,  an  excellent  past  working  relationship  with  two 
of  the  program  management  team  implementation  specialists 
afforded  access  to  all  facets  of  the  Host  program. 
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Exploratory  Case  Study  R«8earch  Method.  Research 
design  can  be  vlewred  from  many  different  perspectives.  The 
most  apparent  perspective  for  this  situation  was  the  degree 
of  problem  crystallization.  Given  this  classification,  it 
was  determined  that  an  exploratory  study  would  best  satisfy 
the  research  objective.  An  exploratory  study  is 
characterized  by  a  loose  structure  and  a  primary  objective 
of  learning.  These  characteristics  allowed  Host  to  be 
evaluated  from  many  different  angles,  and  facilitated  the 
development  of  hypotheses  for  further  analysis  (Emory, 
1985:59).  A  formal  study  was  considered  inappropriate  for 
the  exploration  of  the  Host  program  because  it  required 
precise  procedures  and  data  specifications  (Emory,  1985:60). 

Within  the  scope  of  conducting  an  exploratory  study, 
the  case  study  method  was  selected  to  carry  out  the  effort. 
The  case  study  method  was  selected  based  on  an  evaluation  of 
the  five  main  strategies  outlined  by  Robert  Yin  in  Case 
Study  Research  Design  and  Methods.  The  contemporary  focus 
of  the  case  study  method  allowed  the  use  of  systematic 
interviewing  and  direct  observation  as  sources  of  evidence 
in  addition  to  document  and  artifact  review.  The  ability  of 
a  case  study  to  deal  with  a  full  variety  of  evidence  was  a 
primary  strength  (Yin,  1984:20).  In  addition,  the  case 
study  strategy  did  not  require  manipulation  of  relevant 
behaviors.  This  matched  the  Host  objective  to  minimize 
experimenter  control  of  relevant  behaviors,  and  the  absence 
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of  control  over  Host’s  achievement  of  success.  Finally, 
only  the  case  study  strategy  addressed  the  research 
questions  of  'how*  and  ‘why*  as  operational  links  to  be 
traced  over  time  (Yin,  1984:16,17).  The  ability  of  the  case 
study  strategy  to  support  the  research  requirements  outlined 
above  is  summarized  in  this  quote  from  Robert  Yin’s  book: 

In  brief,  the  case  study  allows  an  investigation 
to  retain  the  holistic  and  meaningful  characteristics 
of  real-life  event8--such  as  Individual  life  cycles, 
and  organizational  and  managerial  processes.  (Yin, 
1984:14) 

Data  Collection  Plan.  As  noted  above,  the  wide  variety 
of  evidence  that  can  be  used  is  a  primary  advantage  of  the 
case  study  strategy.  The  data  collection  plan  used  to 
accomplish  the  case  study  exploration  of  the  Host  computer 
system  included  personal  interviews,  direct  observation,  and 
document  reviews.  These  three  data  collection  techniques 
were  combined  so  that  the  weaknesses  of  each  were  offset  by 
the  strengths  of  the  others  (Tull  and  Hawkins,  1987:115). 

The  following  paragraphs  describe  the  characteristics  and 
application  of  these  three  data  collection  methods. 

Direct  Observation.  Direct  observation  was  used 
to  gather  background  information  on  Host  program  management. 
Corroborating  evidence,  to  either  refute  or  support  that 
gathered  during  the  personal  interviews,  was  also  obtained. 
Direct  observation  was  selected  because  its  flexibility 
allowed  the  observer  to  record  aspects  of  events  and 
behavior  as  they  occurred.  This  data  collection  method  also 
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al loured  the  observer  to  record  xinexpected  situations. 
Increasing  the  quality  of  the  corroborating  evidence 
obtained.  Direct  observation  required  the  observer  to  be 
physically  present  and  to  personally  monitor  what  took 
place,  resulting  In  higher  quality  information  (Emory 
1985: 178)  . 

The  observation  protocol  was  designed  to  support  a 
variety  of  events  and  to  address  the  major  weaknesses 
inherent  in  this  method.  The  events  observed  included 
formal  program  meetings.  Informal  Impromptu  meetings,  a 
telephone  conference,  and  briefings  from  the  program 
management  team  to  the  user.  The  primary  weakness  of  the 
direct  observation  method  was  the  potential  to  overload  the 
observer  with  information  (Emory,  1985:178).  This  weakness 
was  reduced  by  the  following  actions: 

1.  Copies  of  the  briefing  slides  providing  Host 
background  information  were  obtained  and  reviewed. 

2.  A  list  of  the  attendees  was  obtained  for  all 
briefings  and  formal  meetings.  This  list  was  used  to 
corroborate  the  degree  of  teamwork  and  participative 
leadership  used  in  the  Host  program. 

After  obtaining  copies  of  these  records,  the  observer  had 
sufficient  time  and  concentration  to  note  relevant  behavior 
and  attitudes. 

Another  weakness  of  the  direct  observation  method  was 
the  need  for  each  event  to  be  accessible,  and  for  the 
observer  to  be  present  when  the  event  occurs.  This  weakness 
was  countered  by  the  characteristics  of  the  events 
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th«m««lv*8 .  Each  event  had  a  specific  start  and  end  time, 
allowing  the  observer  to  be  present  for  the  duration.  Qiven 
tvrenty  user  sites,  monthly  program  meetings,  and  four  trips 
by  the  observer  to  FAA  HQ,  there  were  multiple  similar 
events  to  select  from  based  on  accessibility.  All  events 
selected  as  necessary  for  observation  were  observed. 

Document  Review.  Document  review,  a  secondary 
data  source,  provided  valuable  detailed  information  on  Host 
contract  elements  cited  by  respondents  as  important  to  Host 
success.  Review  of  'lessons  learned',  advisory  group 
publications,  and  notices  from  both  PAA  Headquarters  and  IBM 
Offices  of  Public  Affairs  also  provided  corroborating 
evidence  for  the  personal  interview  data.  These  documents 
came  from  both  internal  and  external  sources  which  improved 
the  quality  of  the  evidence  drawn  from  them.  Document 
retrieval  was  facilitated  by  the  extensive  library  and 
expert  knowledge  of  Qail  Garwood,  and  by  the  assistance  of 
Michael  E.  Perie,  System  Development  Division  Manager  for 
the  Federal  Aviation  Administration,  Washington  D.C. 

Personal  interview.  The  primary  method  of  data 
collection  used  to  support  the  exploratory  case  study  method 
were  personal  interviews.  William  Emory  defines  personal 
interviews  as: 

A  two-way  conversation  initiated  by  an  interviewer 
to  obtain  information  from  a  respondent.  ...the  topics 
and  pattern  of  discussion  are  generally  dictated  by  the 
interviewer.  (Emory,  1985:160) 


P«PSon«l  interviews  were  selected  because  the  depth  and 


detail  of  information  that  could  be  secured  far  exceeded 
that  for  telephone  and  mail  surveys  (Emory,  1985:160). 
Another  strength  of  the  personal  interview  method  was  its 
versatility.  It  was  the  only  collection  method  that  could 
be  used  to  gather  attitudes  and  opinions  (Emory,  1985:158). 
The  direct  interface  with  the  respondent  increased  the 
quality  of  the  data  by  allowing  clarification  or  additional 
detail  to  be  requested  from  the  respondent  as  necessary.  In 
addition,  the  personal  interview  collection  method  had 
higher  response  rates,  and  more  potential  to  motivate 
respondents  than  the  other  survey  methods.  A  high  response 
rate  was  critical  because  of  the  relatively  small  number  of 
Host  program  management  team  members. 

The  interview  protocol  which  supported  the  Host  program 
exploration  was  designed  to  address  the  major  weaknesses 
inherent  in  the  personal  interview  data  collection  method. 

A  primary  weakness  was  that  data  quality  depended  on  the 
willingness  of  respondents  to  communicate  (Emory,  1985:159). 
This  weakness  was  countered  after  the  initial  Host 
evaluation  indicated  that  program  management  personnel  were 
interested  in  the  research  project,  and  wanted  to  cooperate. 
Another  potential  weakness  was  countered  by  using  the  expert 
knowledge  of  two  Host  program  management  team  specialists  to 
recommend  potential  respondents  who  were  knowledgeable  and 
who  would  provide  reliable  information  (Emory,  1985:159). 
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IntTview  Sample  Selection.  The  goal  was  to  present  a 
comprehensive  list  of  the  management  procedures  and 
organizational  elements  that  contributed  to  Host  success. 
Qail  Garwood  and  Milton  Garwood,  Host  Computer  System 
Planning  and  Implementation  Specialists,  Air  Traffic 
Advanced  Automation  System  Requirements  Branch,  System  Plans 
and  Programs  Division,  Federal  Aviation  Administration  (FAA) 
in  Washington  D.C.,  were  instrumental  in  establishing  the 
criteria  for  selection,  and  in  selecting  the  interview 
sample . 

To  achieve  the  goal  of  composing  a  comprehensive  list, 
all  FAA  Headquarters  branches  directly  involved  in  the 
acquisition  process  were  identified.  A  minimum  of  one 
respondent  was  selected  from  each.  The  criteria  for 
selection  were  number  of  months  of  Host  experience,  and 
degree  of  involvement  with  Host  program  management.  The 
number  of  respondents  selected  from  each  branch  was  related 
to  overall  branch  involvement.  Milt  and  Gail  Garwood 
contributed  greatly  to  the  selection  of  respondents.  They 
assessed  both  the  experience  and  the  involvement  of 
potential  respondents,  and  provided  the  names  and  titles  of 
those  most  qualified.  In  addition,  key  positions  in  FAA 
regional  management,  FAA  ARTCC  staff,  Martin  Marietta 
integration  staff,  and  IBM  national  management  were 
identified.  One  respondent  was  selected  for  each  of  these 
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k«y  positions  identified.  Selection  was  based  on 
accessibility . 

The  ortjanizations  and  ftmctional  areas  sampled  are 
summarized  in  Table  3.1.  As  illustrated  in  this  Table,  key 
organization  identification  and  specific  respondent 
selection  were  conducted  with  the  intent  of  maximizing  the 
variety  of  perspectives  represented,  and  of  increasing  the 
content  validity. 

As  the  interview  process  progressed,  respondents  with 
the  greatest  involvement  in  establishing  and  implementing 
Host  program  management  procedures  were  identified.  These 
key  respondents  were  interviewred  several  times  to  increase 
data  reliability.  The  28  individuals  interviewed  during  the 
exploratory  study  of  Host  are  listed  in  Appendix  A:  Host 
Computer  Program  Interviewees. 

Interview  Protocol .  A  primary  weakness  of  personal 
interviewing  is  the  degree  of  bias  that  can  be  introduced  by 
the  interviewer.  An  interview  protocol  was  defined  and 
implemented  to  counter  this  weakness,  and  to  reduce  response 
deviations  due  to  nonstandard  interviewing  procedures. 

Before  implementation,  the  interview  protocol  was  reviewed 
by  Milt  and  Oail  Qarwood  and  Colonel  Delaney.  In  Business 
Research  Methods .  William  Emory  listed  the  following  broad 
criteria  for  a  successful  personal  interview  (Emory.  1985: 
161)  ; 

1.  Accessibility  by  the  respondents  to  needed 
information . 
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Table  3.  1 


SiiiBinary  of  Sample  Population  -  Host  Computer  Program 


Total  Number 
Interviewed 

Federal  Aviation  Administration  (FAA) 


Organizations  Sampled  and 
Fxinctional  Areas  Represented 


Air  Traffic  Plans  and  Requirements  Service  1 

System  Development  Division  1 

Contracting  Officer  1 

Contracting  Officer  Technical  Representative  1 

Headquarters  Program  Management  Team  Participants 

Information  Systems  Branch  1 

Software  Requirements  Branch  1 

Computer  Complex  Program  Branch  1 

System  Requirements  Branch  2 

Host  CoB^uter  System  Branch  3 

Host  Systems  Implementation  Branch  5 

Total  13 

Regional  Program  Management  Team  Participants 
Airway  Facilities  Division 

Air  Traffic  Plans  and  Programs  Division  1 

Automation  Engineering  2 

Total  4 

Air  Route  Traffic  Control  Center,  End  User  2 


Total  FAA  Personnel  Interviewed  23 

Martin  Marietta  Corp. 

Regional  Integration  1 


International  Business  Machines  (IBM)  Corp. 

Host  Computer  System: 

Program  Service  Executive  1 

Repairability ,  Maintainability  and  Availability  1 
Field  Deployment  1 

Physical  Planning  1 


Total  IBM  Personnel  Interviewed  4 


Total  Number  of  Individuals  Interviewed 


28 
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2.  An  understanding  by  the  respondents  of  their  roles. 

3.  Motivation  of  the  respondent  to  accept  such  a  role 
and  to  fulfill  its  requirements. 

The  interview  protocol  developed  for  Host  was  based  on  these 
criteria.  A  summary  of  that  protocol  is  outlined  below. 

1.  Accessibility  by  the  respondents  to  needed 
information  was  addressed  by  sample  selection  procedures. 
Only  individuals  who  met  the  selection  criteria  were 
interviewed . 

2.  Qail  Garwood  personally  contacted  each  selected 
individual  to  set  up  an  interview  appointment.  At  that 
time  she  briefly  covered  the  purpose  of  the  research  and  the 
role  expected  of  the  respondent.  Arranging  all  Interviews 
through  Qail,  a  well  known  and  highly  regarded  Host  program 
management  team  member,  increased  respondent  motivation  to 
accept  the  role.  It  also  lent  credibility  to  both  the 
research  and  the  researcher. 

3.  In  all  cases,  either  Milt  or  Gall  Garwood  briefly 
introduced  the  interviewer  to  the  respondent,  and  then 
departed  before  the  data  collection  began.  This  personal 
introduction  provided  evidence  to  respondents  that  the 
research  was  sanctioned  by  FAA  management,  and  reassured 
them  that  the  purpose  of  the  Interview  was  to  gather 
information  versus  to  audit.  The  introduction  also  relaxed 
and  motivated  the  respondent  while  increasing  the 
credibility  of  the  interviewer.  Milt  and  Qail  departed 
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aftar  the  Introduction  ao  that  the  respondent  could  answer 
freely  and  without  influence. 

4.  Continuing  with  the  introduction,  the  respondents 
were  told  that  their  names  and  positions  would  be  recorded 
with  their  interview  responses.  They  were  then  given  the 
opportunity  to  decline  the  interview.  All  respondents 
expressed  the  desire  to  continue.  The  lack  of  anonymity  did 
not  adversely  affect  the  quality  of  the  responses  because 
the  research  objective  and  interview  questions  ware  of  a 
positive  nature,  and  were  not  threatening. 

5.  To  conclude  the  Introduction,  the  research 
objectives  were  explained  to  outline  the  role  of  the 
respondent,  and  to  spark  the  respondent's  interest.  The 
respondent  was  informed  that  the  kind  of  answer  sought  was 
their  opinion  or  perception.  The  program  management 
experience  of  the  interviewer  was  also  addressed  so  that  the 
respondent  would  have  an  idea  how  complete  each  answer 
should  be. 

6.  During  the  interview,  the  opening  questions  were 
read  directly  from  the  'interview  session  organization' 
sheet  to  increase  interview  standardization.  The 
interviewer  requested  clarification  of  responses  and  ■ 
additional  information  from  the  respondent  as  necessary. 
These  subsequent  questions  provided  minimal  guidance,  and 
were  carefully  worded  so  that  the  respondent  would  not  be 
'lead*  to  a  specific  answer. 
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Th«  interview  quAStions  vwre  general  and  open  ended  to 


match  the  intent  of  an  exploratory  study.  According  to 
William  Emory,  unstructured  in-depth  interviews  are  often 
used  in  exploratory  research,  especially  in  case  research 
among  various  participants  in  a  major  event  (Emory, 

i 

1985:203).  When  conducting  unstructured  in-depth  interviews 
the  interviewer’s  task  is  to  encourage  the  respondent  to 
talk  about  a  set  of  topics  given  minimal  prompts  and  guiding 
questions  (Emory,  1985:203).  The  interview  questions  posed 
to  the  acquisition  program  management  team  and  contractor 
representatives  were: 

1.  Do  you  believe  that  Host  was  a  successful 
program? 

2.  Why  do  you  believe  that  it  was  (or  was  not) 
successful? 

3a.  If  the  subject  did  not  believe  that  Host  was  a 
successful  program:  What  do  you  perceive  prevented 
this  success? 

3b.  If  the  subject  did  believe  that  Host  was  a 
successful  program:  Why  do  you  think  that  Host  was 

successful?  Added  for  clarification:  What  do  you 
perceive  that  the  program  management  team  did  to 
achieve  this  success? 

7.  Responses  were  recorded  as  they  were  given.  This 
reduced  interviewer  bias  and  ensured  accurate  recording. 

Observation  Sample  Selection.  For  each-  Air  Route 
Traffic  Control  Center  (ARTCC) ,  the  Host  computer  system 
implementation  plan  outlined  seven  key  events  to  be 
accomplished  at  staggered  intervals.  Of  those  seven  events, 
four  marked  the  completion  of  testing  and  validation 
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procsdures .  Due  to  the  minimal  activity  to  be  observed, 
these  four  events  were  eliminated  from  consideration  for 
selection:  1.  hardware  delivery,  2.  initial  operating 
capability,  3.  operational  readiness  demonstration,  and  4. 
government  acceptance. 

During  the  remaining  three  events  numerous  activities 
were  accomplished.  In  addition,  these  events  involved  all 
ARTCC  members,  and  contained  the  contributions  of  program 
management  team  members  from  both  FAA  Headquarters  and  IBM 
Federal  Systems  Division.  A  description  of  these  three 
events  follows  (Martin  Marietta,  1986:5-20): 

1.  The  Preliminary  ABTCC  Site  Survey,  held  six  months 
before  Host  hardware  delivery.  Program  management  team 
representatives  to  presented  informative  briefings, 
addressed  site  member’s  concerns  and  assigned  action  items 
to  prepare  for  hardware  delivery. 

2.  The  Second  ARTCC  Site  Survey,  held  one  month  before 
Host  hardware  delivery.  Program  management  team 
representatives  presented  a  more  detailed  informative 
briefing,  gathered  additional  feedback,  ensured  that  all 
action  items  were  accomplished,  identified  the  contractor 
site  team,  and  ensured  that  both  the  facilities  and 
personnel  were  prepared  for  hardware  delivery. 

3.  Dedication  of  the  Host  Computer  System.  A  major 
event  attended  by  state  dignitaries,  program  management  team 
representatives,  FAA  top  management  and  all  ARTCC  members. 
This  event  signaled  final  success.  The  site  had  accepted 
and  was  using  the  Host  Computer  System. 

Due  to  the  numerous  activities  and  participants 
available  for  observation,  these  three  events  were 
considered  for  selection.  The  Preliminary  Site  Survey  was 
eliminated  from  consideration  for  observation  because  all  20 
ARTCCs  had  passed  this  event.  The  Indianapolis  ARTCC  was 
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selected  as  the  location  to  observe  the  Second  Site  Survey. 
The  Cleveland  ARTCC  was  selected  as  the  location  to  observe 
a  Host  Computer  System  Dedication.  Both  of  these  sites  were 
selected  based  on  accessibility  and  date  constraints. 
According  to  several  FAA  program  management  team  members , 
events  at  these  two  sites  were  representative  of  all  sites. 

The  other  meetings  or  events  attended  to  gather 
corroborating  evidence  were  selected  based  on  accessibility 
and  interview  schedule  constraints.  Appendix  B:  Host 
Computer  Program.  Events  Observed,  outlines  the  events 
observed  during  the  exploratory  study  of  Host.  These  events 
are  listed  in  chronological  order. 

Observation  Protocol.  An  observation  protocol  with 
minimal  standardization  was  selected  because  it  best 
benefited  the  heuristic  nature  of  the  exploratory  study 
(Emory,  1985:179).  According  to  William  Emory  in  Business 
Research  Methods  the  observation  protocol  must  answer  the 
question  of  who,  where,  when,  how,  and  what  within  the 
context  of  minimal  standardization  (Emory,  1965:181).  In 
addition,  the  observer-subject  relationship  and  observation 
setting  must  be  addressed  (Emory,  1965:178-181). 

Who,  required  identification  of  the  subject  to  be 
observed  (Emory,  1985:181).  All  of  the  events  selected  for 
direct  observation  involved  a  specific  and  finite  group  of 
participants.  The  behavior  and  attitudes  of  all 
participants  were  relevant,  and  provided  corroborating 
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evidence  for  evaluating  the  personal  interview  data.  The 
subjects  selected  for  observation  within  each  event  were  the 
participants,  including  meeting  leaders  or  briefers. 

Where,  required  identification  of  the  sites  to  be  used 
for  observation.  The  selection  of  these  sites  was  discussed 
in  the  preceding  subsection. 

When,  required  that  the  importance  of  the  time  of 
observation  be  addressed  (Emory,  1985:182).  This  was  not  a 
relevant  issue  because  each  event  was  attended  from  start  to 
finish. 

How,  required  that  the  process  of  recording  the  data, 
and  method  of  observation  be  addressed  (Emory,  1985:182). 

The  method  of  observation  selected  was  direct  and  simple,  as 
outlined  previously  in  the  subsection  on  data  collection. 

The  data  was  recorded  as  it  was  observed.  The  next  section 
addresses  the  content  specification. 

What,  required  that  those  specific  conditions,  events, 
and/or  activities  to  be  observed  be  identified  (Emory, 
1985:182).  The  goal  was  to  collect  evidence  to  either 
corroborate  or  refute  the  data  collected  from  the  personal 
interviews.  Both  factual  and  inferential  variables  were  of 
interest.  The  following  factual  variables  were  selected  for 
observation:  1.  the  general  category  of  the  issues  discussed 
or  the  information  presented,  2.  the  participants, 
categorized  by  functional  organization  or  program  management 
position,  and  3.  the  actions  or  status  reports  resulting 


from  the  feedback  received  from  users.  Observation  of  these 
variables  was  used  to  Identify  communication  channels,  to 
determine  degree  of  participation  and  user  Involvement,  and 
to  assess  program  management  teamwork.  The  following 
Inferential  variables  were  selected  for  observation:  1.  the 
degree  of  cohesion  and  agreement  among  briefers,  2.  the 
enthusiasm  of  the  participants  and/or  users,  3.  the 
receptiveness  of  the  briefers  or  meeting  leaders  to 
feedback,  and  4.  the  general  attitude  of  the  briefing  or 
meeting  host.  Observation  of  these  variables  was  used  to 
assess  program  management  teamwork,  commitment  to  user 
Involvement,  and  user  satisfaction  with  both  the  Host 
Implementation  process  and  the  computer  system. 

The  observation  protocol  addressed  the  observer-subject 
relationship  In  terms  of  directness  of  observation,  observer 
participation,  and  observer  concealment.  As  previously 
discussed,  the  direct  method  best  met  the  need  to  obtain 
corroborating  evidence.  The  observer  did  not  participate  In 
the  event  under  observation.  This  was  In  accordance  with 
case  study  method  requirements.  The  observer  sat  behind  the 
participants  with  a  FAA  Headquarters  program  management  team 
member,  and  noted  the  attitudes  and  behavior  of  the 
participants.  Milt  Qarwood ,  Gall  Garwood,  and  Captain 
Kenneth  Jennings,  an  expert  In  management  and  behavior  In 
organizations,  all  believed  that  It  was  not  necessary  to 
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conc«al  the  observer.  They  believed  that  there  was  minimal 
risk  of  atypical  activity  by  the  subjects  due  to  observer 
presence  because  the  observer  posed  no  threat  and  blended  in 
with  the  numerous  participants. 

Data  Analysis.  After  the  interviews  were  conducted, 
the  responses  of  each  respondent  were  formally  documented. 
The  management  procedures  and  organizational  elements 
perceived  by  the  respondents  to  be  vital  for  achieving  Host 
success  were  initially  reviewed  to  extract  the  basic 
concepts  underlying  the  elements.  Previous  study  of 
classical  management  theory,  organizational  dynamics,  and 
the  role  of  management  in  organizations  was  vital  to 
identifying  these  underlying  concepts.  The  elements  were 
then  reviewed  in  detail  and  categorized  according  to  the 
basic  concepts  identified.  The  recommendations  of  previous 
research  to  achieve  program  success  were  used  to  further 
categorize  and  analyze  the  elements.  The  resulting 
management  strategy  was  composed  of  six  management 
categories,  each  supported  by  five  to  six  elements.  This 
management  strategy  was  then  evaluated  by  DOD  acquisition 
experts  to  answer  the  investigative  questions  supporting  ti.e 
second  research  objective.  The  analysis  process  and 
subsequent  results  from  the  exploratory  case  study  of  Host 
are  presented  in  Chapter  IV. 
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specific  Procedwa  for  Anawerinjj  Beaearch  Objective  II 

The  exploratory  case  study  research  conducted  to  meet 
the  first  research  objective  resulted  in  a  management 
strategy  hypothesized  to  aid  DOD  program  managers  to  achieve 
success.  In  Exploring  Marketing  Besearoh.  by  William 
Zikmund,  the  researcher  was  cautioned  not  to  stop  after  the 
initial  exploratory  stage.  Mr.  Zikmund  emphasized  the  need 
to  analyze  the  attributes  identified  during  exploration 
before  implementation  to  reduce  the  risk  of  erroneous 
implementation  (Zikmund,  1985:118).  To  meet  the  second 
research  objective.  DOD  acquisition  experts  analyzed  the 
elements  identified  during  the  exploratory  case  study  of  the 
Host  computer  system. 

Investigative  question  5  addressed  the  need  for 
analysis  and  was  answered  by  data  collected  from  the  expert 
opinion  method.  Experts  in  the  DOD  acquisition  community 
were  asked  to  determine  which  of  the  hypothesized  elements 
formed  the  management  strategy  that  would  potentially 
increase  the  DOD  acquisition  success  rate.  A  progressive 
series  of  supporting  questions  was  used  to  facilitate  this 
determination.  First,  sub-question  5a  asked  each  expert  to 
identify  which  elements  contained  new  concepts.  The  next 
iteration,  met  by  sub-questions  5b  and  5c,  asked  the  experts 
to  identify  the  elements  necessary  for  program  success 
regardless  of  application  or  current  regulation,  and  then  to 
categorize  the  elements.  Each  expert,  according  to  his/her 
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ovm  •xperiance,  catagorlzed  the  elements  as  either  currently 
used  In  DOD,  not  used  but  potentially  applicable,  or  not 
used  and  not  applicable.  The  third  series  of  sub-questions 
was  contingent  on  the  category  assigned  by  the  expert  to 
each  element.  For  the  elements  identified  as  potentially 
applicable  to  DOD,  sub-question  5d  asked  the  expert  to 
identify  what  had  to  be  done  before  the  element  could  be 
implemented.  For  the  elements  identified  as  not  applicable 
to  DOD,  sub-question  5e  asked  the  expert  to  explain  why. 
Finally,  for  the  elements  identified  as  currently  applied  or 
potentially  applicable  to  DOD,  sub-question  5f  asked  the 
expert  to  assess  the  range  of  programs  for  which  the  element 
was  appropriate. 

The  final  step  necessary  to  satisfy  the  second  research 
objective  was  analysis  of  the  data  collected.  Evaluation 
and  tabulation  of  the  expert  responses  to  investigative 
question  5,  and  its  six  sub-questions,  met  this  need  and  are 
presented  in  Chapter  V. 

The  following  sections  address  the  specific  areas 
composing  the  methodology  outlined  above.  First,  the  expert 
opinion  research  method  is  defined.  Next,  the  interview 
sample  selection  is  discussed,  followed  by  an  outline  of  the 
interview  process  protocol.  Finally,  the  method  of  data 
analysis  is  presented. 

Expert  Opinion  Research  Method.  Again,  an  exploratory 
study  was  most  appropriate  for  meeting  the  research 


objective.  The  exploratory  research  process  used  so  far  has 
progressively  narrowed  the  scope  of  the  research  problem  and 
identified  attributes  for  investigation.  The  purpose  of  the 
exploratory  research  conducted  to  meet  the  second  research 
objective  is  to  screen  alternatives  (Zikmund,  1985:101). 

The  elements  hypothesized  to  compose  the  management  strategy 
that  will  help  DOD  program  managers  increase  program  success 
rates  were  screened,  or  evaluated,  to  determine  which  were 
feasible.  This  step  in  the  research  process  was 
accomplished  with  an  experience  survey. 

An  experience  survey  was  conducted  to  get  information 
that  would  provide  the  insight  necessary  to  sharpen  the 
concepts  already  identified  (Emory,  1985:63;  Zikmund, 
1985:43).  Mr.  Zikmund  defined  an  experience  survey  as  a 
process  of  discussing  concepts  with  knowledgeable  managers 
who  have  had  personal  experience  in  the  field  to  obtain 
insights  into  the  relationships  among  variables  (Zikmund, 
1985:44,102).  William  Emory  explained  that  the  researcher 
would  profit  from  using  experience  surveys  because 
‘published  data  are... seldom  more  than  a  fraction  of  the 
existing  knowledge  in  a  field  of  interest ...  and  persons 
experienced  in  the  area  of  study  can  provide  insight- 
stimulating  information*  (Emory,  1985:63) . 

Experience  surveys  can  be  very  informal,  or  slightly 
structured  (Zikmund,  1985:103).  The  need  to  ensure  content 
validity  and  data  reliability  required  the  development  and 


in^lamentatlon  of  an  Interview  protocol  to  provide  some 
structure.  In  addition,  selection  criteria  were  defined  and 
used.  In  accordance  with  literature  on  research 
methodology,  the  experience  survey  involved  a  small  number 
of  interviews  with  experienced  people  who  were  carefully 
selected  (Zikmund,  1985:103).  The  interview  protocol  was 
developed  following  this  methodology  guidance.  Few  formal 
questions  were  asked,  and  the  expert  was  allowed  to  discuss 
the  questions  with  few  constraints.  Respondents  were 
selected  baaed  on  qualifications,  rather  than  by  a 
representative  probability  sample  (Zikmund,  1085:103). 

Interview  Sample  Selection.  Lt  Col  Qary  Delaney,  Chief 
of  the  Defense  Resources  Management  Division,  Department  of 
System  Acquisition  Management,  School  of  Systems  and 
Logistics,  Air  Force  Institute  of  Technology,  located  at 
Wr ight-Patterson  AFB,  OH,  was  instrumental  in  establishing 
the  criteria  for  selection,  and  in  selecting  the  interview 
sample.  The  goal  was  to  identify  DOD  acquisition  experts. 
For  the  purpose  of  achieving  this  research  objective, 
experts  were  defined  by  the  breadth  of  their  program 
management  experience,  and  by  the  perspective  afforded  by 
their  position.  It  was  determined  that  to  obtain  reliable 
data  on  the  validity  and  applicability  of  the  hypothesized 
program  management  strategy,  an  expert  should  have  program 
management  experience  over  several  programs.  In  addition, 
it  was  determined  that  the  overall  perspective  of  a  DOD 


policy  maker  for  program  management  was  desired.  An  expert 
working  in  a  policy  making  position  would  have  the  broad 
perspective  and  information  access  necessary  to  determine 
each  strategy  element’s  contribution  to  acquisition  success, 
to  define  element  applicability,  and  to  identify  any 
shortfalls.  For  expert  selection,  the  first  consideration 
was  breadth  of  experience,  second  was  position,  and  third 
was  accessibility. 

It  was  determined  that  each  of  the  following  offices 
were  potential  locations  for  the  experts  that  would  meet  the 
criteria  established  above,  and  should  be  represented  in  the 
interview  sample:  1)  Office  of  the  Secretary  of  the  Air 
Force,  2)  Office  of  the  Secretary  of  the  Air  Force,  3) 
Headquarters  Air  Force  Systems  Command  Staff,  and  4)  Defense 
Systems  Management  College  staff.  The  respondents  were 
selected  on  the  basis  of  recommendation.  Lt  Col  Delaney 
telephoned  personal  contacts  in  each  of  the  areas  selected, 
described  the  purpose  of  this  research,  explained  the 
criteria  for  expert  selection,  and  obtained  the  names  of 
individuals  who  the  contact  believed  would  satisfy  the 
criteria.  Table  3.2  lists  the  titles  of  the  eleven 
individuals  interviewed. 

Interview  Protocol.  The  Interview  protocol  for 
conducting  the  expert  interviews  was  based  on  the  three 
criteria  for  a  successful  interview  previously  discussed. 


Table  3.2 


Summary  of  Sample  Population  -  DOD  Acquisition  Experts 


Organizations  Sampled  and 
Titles  of  Individuals  Interviewed 


Office  of  the  Secretary  of  Defense.  Under  Secretary  of 
Defense  (Acquisition) .  Directorate  (Program  Integration) 
Director 

Office  of  the  Assistant  Secretary  of  the  Air  Force 
Deputy  Assistant  Secretary  of  the  Air  Force 
(Acquisition  Management  and  Policy) 

Directorate  of  Contracting  and  Manufacturing  Policy 
Associate  Director 

Directorate  of  Program  Plannind  and  Integration 
Director 

Headquarters  Air  Force  Systems  Command 

Principal  Assistant  to  the  Deputy  Chief  of  Staff, 
Contracts 

Deputy  Chief  of  Staff, 

Technology  and  Requirements  Planning 
Chief  , 

Programs,  Analysis  and  Initiatives  Division 
Inspector  General 

Defense  Systems  Management  College 
Course  Director  for  Policy 
Professor  of  Engineering  Management 

The  Analytic  Sciences  Corporation 

Manager , 

Defense  Acquisition  Management  Department 


isssssss: 


Total  Number  of  Individuals  Interviewed 


11 
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Before  implementation,  the  interview  protocol  was  reviewed 
by  Lt  Col  Delaney,  and  the  interview  questions  were  reviewed 
by  several  faculty  members.  The  protocol  followed  while 
conducting  each  interview  is  contained  in  Appendix  E:  POD 
Expert  Interview.  Interview  Session  Organization.  A 
description  of  the  protocol  development  follows. 

1.  Each  selected  individual  was  contacted  by  letter 
and  asked  to  grant  a  30-45  minute  interview.  The  letters 
were  signed  by  Lt  Col  Delaney,  and  explained  the  purpose  of 
AFIT,  and  the  purpose  and  methodalogy  of  the  research. 
Attached  to  the  letter  was  a  brief  fact  sheet  on  the  Host 
Computer  System  to  familiarize  the  reader  with  the  program, 
and  to  justify  why  Host  was  considered  to  be  a  success.  The 
Colonel's  signature  and  AFIT  letterhead  lent  legitimacy  to 
the  study.  The  information  attached  motivated  the 
respondent  to  participate.  A  sample  of  the  initial  letter 
is  contained  in  Appendix  C:  POD  Expert  Interview.  Request 
For  Interview  Letter.  Appendix  D:  POD  Expert  Interview. 

Host  Program  Fact  Sheet  contains  a  sample  of  the  letter 
attachment . 

Of  the  twelve  individuals  contacted,  eleven  granted  an 
interview  and  one  declined  because  of  schedule  conflict.  A 
replacement  with  equivalent  position  and  experience  was 
selected,  contacted,  and  agreed  to  participate.  Another 
respondent  was  later  eliminated  after  it  was  decided  that  he 
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did  not  me«t  the  experience  criteria.  All  interviews  were 
conducted  in  the  respondent’s  office. 

2.  The  interview  be^an  with  a  brief  statement  about 
the  purpose  and  methodology  of  the  research  to  refresh  the 
respondent’s  memory  and  to  increase  interest  in  the  project. 
Next,  respondents  were  asked  for  permission  to  include  a 
brief  summary  of  the  interview,  including  name  and  position, 
in  an  appendix  to  the  thesis.  This  lack  of  anonymity  did 
not  adversely  affect  the  quality  of  the  responses  because 
the  interview  questions  were  not  of  a  personal  nature,  and 
because  the  respondents  were  given  the  opportunity  to  edit 
the  resulting  paraphrase.  The  objective  of  the  interview, 
the  proposed  agenda,  and  an  overview  the  respondent’s  role 
were  then  covered. 

3.  After  the  introduction,  each  respondent  was  asked 
if  the  interviewer  could  share  with  them  some  of  the  reasons 
that  the  Host  program  management  team  perceived  that  Host 
was  successful,  and  if  the  respondent  would  use  his/her 
expert  perception  to  determine  if  there  was  anything  new,  or 
potentially  applicable  to  the  way  DOD  manages  programs.  The 
respondent  was  asked  if  they  had  any  questions  or  comments 
about  their  role,  the  research  methodology  or  the  Host 
computer  system  acquisition.  These  actions  ensured  that  the 
respondent  understood  their  role,  and  also  provided  some 
motivation  to  participate. 
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4.  Having  completed  the  introduction  and  established  a 
rapport  with  the  respondent,  the  interview  protocol  focused 
on  gathering  data.  A  summary  of  the  elements  hypothesized 
to  form  the  management  strategy  for  achieving  program 
success  was  handed  to  the  respondent.  The  source  of  the 
elements  and  the  development  of  the  six  categories 
containing  them  was  briefly  explained.  A  copy  of  this 
handout  is  contained  in  Appendix  F:  POD  Expert  Interview. 
Reasons  For  Host  Success. 

5.  The  process  of  addressing  each  of  the  six 
categories  began  after  the  respondent  was  given  a  few 
minutes  to  review  the  proposed  management  strategy.  For 
each  category,  an  explanation  of  the  general  area  was 
followed  by  a  brief  overview  of  why  the  Host  program 
management  team  believed  that  the  area  contributed  to 
achieving  program  success.  After  the  category  introduction, 
the  respondent  was  asked  to  answer  specific  questions  about 
each  element. 

The  interview  questions  were  designed  to  support 
unstructured  in-depth  interviews.  This  category  of 
interviews  was  selected  because  it  was  most  appropriate  for 
dealing  with  complex  topics  (Emory,  1985:203).  The 
interview  questions  were  open  ended  to  allow  the 
respondents,  each  an  acquisition  expert,  to  address  the 
areas  they  believed  to  be  relevant  to  the  strategy 
evaluation.  This  corresponds  to  the  purpose  of  using  an 
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oxpert  opinion  methodology.  The  areas  addressed  by  each 
question  corresponded  to  the  final  investigative  question. 
Each  respondent  was  asked  the  following  questions  for  each 
element : 


a.  Does  this  element  contain  a  concept  that  is 
new  to  DOD  program  management? 

b.  Is  this  element  necessary  for  an  acquisition 
program  to  be  successful?  If  not:  Why? 

If  the  answer  to  question  b  was  affirmative,  then 
the  following  questions  were  also  asked  about  the 
element : 

c.  Is  this  element  currently  used  or  potentially 
applicable  to  the  DOD  acquisition  process? 

1)  If  not:  Why? 

2)  If  yes:  What  is  the  range  of  programs  to 
which  this  element  applies? 

3)  If  not  currently  used,  but  potentially 

applicable:  What  must  be  done  before  this  element 

can  be  implemented? 

The  interviewer  prompted  the  respondent  for  details  or 
clarification  as  necessary.  Each  individual  was  asked  the 
same  questions,  and  subsequent  leading  questions  were 
avoided.  Each  respondent  determined  which  elements  they 
were  qualified  to  evaluate. 

6.  Responses  were  recorded  as  they  were  given  to 
reduce  interviewer  bias  and  ensure  accurate  recording.  To 
provide  a  method  for  quickly  referencing  the  element 
corresponding  to  respondent  comments,  each  element  was 
assigned  a  unique  two  character  identifier  and  listed  on  a 
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master  outline.  This  master  outline  was  used  to  record  each 
interview. 

7.  After  all  of  the  categories  had  been  discussed,  the 
respondent  was  asked  to  both  verify  and  clarify  the  main 
points  recorded.  This  increased  content  validity.  In 
closing,  respondents  were  informed  that  a  paraphrase  of 
their  comments  would  be  sent  within  seven  days  for  their 
review  and  annotation.  The  respondents  were  also  asked  for 
permission  to  include  the  reviewed  paraphrase  of  their 
comments  in  an  appendix  to  the  thesis,  and  direct  quotes 
from  that  paraphrase  in  the  analysis  of  the  management 
strategy . 

8.  The  interview  was  concluded  by  thanking  the 
respondent  for  his/her  time. 

9.  Respondents  all  received  an  interview  follow-up 
package.  The  package  included  a  letter  thanking  the 
respondents  for  their  time  and  specific  contributions,  and  a 
paraphrase  of  the  interview.  The  respondent  was  asked  to 
edit,  adjust  or  comment  directly  on  the  draft  as  necessary. 
The  letter  also  reminded  the  respondent  that  the  edited 
interview  paraphrase  would  be  included  in  an  appendix  to 
this  thesis,  and  direct  quotes  may  be  included  in  the  body. 
The  ‘Host  Fact  Sheet*  and  ‘Menus  for  Success'  were  enclosed 
to  provide  easy  access  to  any  background  information  that 
the  respondent  could  require.  A  stamped  and  addressed 
envelope  was  provided  to  facilitate  the  return  process.  To 


avoid  suspensing  the  respondent  while  meeting  time 
constraints  the  letter  stated  that  no  response  would  be 
interpreted  as  acceptance  of  the  interview  paraphrase.  A 
sample  follow-up  package  is  included  in  Appendix  G:  POD 
Expert  Interview.  Thank  You  and  Request  For  Review  Letter. 

Data  Analysis.  After  the  interviews  were  conducted, 
the  comments  of  each  respondent  were  formally  documented. 

The  interview  paraphrases  were  then  submitted  to  each 
respondent  for  review.  Nine  of  the  eleven  respondents 
edited  and  returned  their  interview  paraphrases.  These 
replaced  the  initial  draft  paraphrases  and  were  used  as  the 
data  source  for  further  analysis.  The  analysis  was 
conducted  element  by  element.  The  comments  and  responses 
for  each  element,  across  all  interviews,  were  grouped 
together.  The  degree  of  cohesion  between  respondents  as  to 
the  importance  and  applicability  of  each  element  was  noted. 
Next,  the  investigative  sub-questions  were  answered  for  each 
element.  The  answers  indicated  which  elements  should  remain 
in  the  management  strategy,  and  which  should  be  eliminated. 

In  almost  every  interview  the  respondent  suggested 
additional  elements  that  he/she  believed  were  critical  for 
program  success.  These  suggestions  were  listed  and  the 
frequency  of  suggestion  was  tabulated.  The  Host  contracting 
officer  technical  representative,  Arthur  Simolunas,  and  the 
Air  Traffic  Plans  and  Requirements  Service  Director, 

Theodore  Beckloff,  were  informally  interviewed  on  21  July 


1988.  The  purpose  of  these  Interviews  was  to  obtain  answers 
to  the  questions  raised  during  the  DOD  expert  opinion 
interviews  and  to  evaluate  the  additional  suggested 
elements.  Based  on  the  responses  of  these  two  high  level 
managers  within  the  Host  program,  some  of  the  suggested 
elements  were  added  to  the  management  survey.  The  result 
was  a  program  management  strategy  believed  by  DOD 
acquisition  experts  to  increase  the  acquisition  success 
rate.  The  analysis  process  is  presented  in  Chapter  V. 

Summary 

A  successful  contemporary  acquisition  program  was 
selected  for  initial  analysis  by  this  research.  The  Federal 
Aviation  Administration  Host  Computer  System  was  selected 
because  it  met  or  exceeded  all  criteria  of  success 
identified  in  the  literature  review,  and  because  it  was 
accessible.  The  exploratory  case  study  of  the  Host  program 
resulted  in  the  identification  of  the  management  procedures 
and  organizational  elements  perceived  by  the  Host  program 
management  team  to  contribute  to  Host  success.  Analysis  of 
these  elements  in  conjunction  with  the  recommendations 
presented  in  the  literature  review  culminated  in  a 
hypothesized  management  strategy  to  assist  DOD  program 
managers  achieve  success.  Chapter  IV  evaluates  the  results 
of  the  exploratory  case  study  and  outlines  the  analysis 
process . 
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The  expert  opinion  research  method  was  used  to  analyze 
the  hypothesized  management  strategy.  The  results  of  the 
expert  survey  are  reported  and  analyzed  in  Chapter  V. 

In  reviewing  the  research  methodology  presented  and  the 
data  evaluation  outlined  in  following  chapters,  it  is 
important  to  remember  the  limitations  of  the  exploratory 
research  techniques.  First,  conclusions  based  on 
qualitative  data  may  be  subject  to  considerable  interpreter 
bias  (Zikmund,  1985:117).  This  limitation  was  minimized  by 
using  rigorous  interview  protocols,  and  by  limiting 
interpretation  of  data  to  documented  respondent  comments. 
Second,  ’Exploratory  research  can  not  deliver  what  it  does 
not  promise'  (Zikmund,  1985:118).  'Exploratory  research  is 
not  intended  to  provide  conclusive  evidence ...  usual ly  this 
type  of  research  is  conducted  with  the  expectation  that 
subsequent  research  will  be  conducted  if  conclusive  evidence 
is  required'  (Zikmund,  1985:35).  The  management  strategy 
resulting  from  this  research  effort  has  been  evaluated  by 
experts,  but  if  conclusive  or  causal  evidence  is  required, 
then  formal  causal  research  must  be  conducted. 
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IV.  Host  Exploration  Findings  and  Analysis 

Chapter  overview 

This  chapter  presents  a  discussion  of  the  data 
collected  during  the  exploratory  case  study  of  the  Federal 
Aviation  Administration's  Host  computer  program.  The 
findings  and  analysis  are  divided  Into  ten  sections.  First, 
the  research  question  and  supporting  Investigative  questions 
to  be  answered  in  this  chapter  are  restated.  The  second 
section  presents  a  brief  description  of  the  Host  computer 
system.  The  subsequent  section  provides  the  results  by 
%dilch  Host  was  determined  to  be  successful.  The  next  six 
sections  discuss  the  responses  received  to  the  Interview 
questions,  analyze  those  responses,  and  present  the 
findings.  These  sections  contain  the  answer  to 
investigative  question  3b.  The  final  section  compares  the 
data  collected  during  the  exploratory  case  study  with  the 
recommendations  of  previous  research  reported  In  Chapter  II. 
The  answer  to  Investigative  question  4  Is  then  presented. 

Research  Objective  and  Supporting  Investigative  Questions 

The  Federal  Aviation  Administration's  Host  computer 
program  was  selected  as  the  successful  program  for 
exploration.  In  response  to  Investigative  question  3a,  the 
Host  computer  program  was  found  to  meet  all  of  the  criteria 
for  a  successful  program.  The  analysis  supporting  this 
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finding  was  accomplished  in  the  Successful  Acquisition 
Program  section  of  Chapter  III.  Representatives  from  each 
component  of  the  Host  program  management  team  were 
interviewed  to  identify  which  managerial  procedures  and 
organizational  elements  they  perceived  to  contribute  to  Host 
success.  After  comparing  these  elements  with  the 
recommendations  of  current  studies,  a  management  strategy 
for  achieving  DOD  program  success  was  hypothesized.  This 
analysis  provided  the  data  to  achieve  the  first  research 
ob j  active . 

Research  Objective  I.  To  discover  and  analyze  the 
elements  of  success  used  to  manage  a  successful  acquisition 
program. 

Investigative  Question  3.  What  are  the  management 
procedures  and  organizational  elements  used  by  the  program 
management  team  of  a  successful  development  type 
acquisition?  This  investigative  question  is  answered 
through  the  following  two  sub-questions: 

a.  Does  the  program  meet  the  criteria  established  in 
the  literature  review  for  a  successful  acquisition? 

b.  Of  the  elements  used,  which  are  perceived  by  the 
program  management  team  to  be  vital  to  the  success  of  their 
program? 

Investigative  Question  4.  How  do  the  management 
procedures  and  organizational  elements  recommended  in 
previous  research  compare  to  those  perceived  by  the  program 
management  team  to  be  vital  to  the  success  of  their  program? 
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Boat  D^acription 

Automation  waa  introduced  into  the  air  traffic  system 
in  the  early  1970s.  The  first  automation  program  processed 
flight  plan  information  and  prepared  flight  progress  strips. 
Later,  the  program  was  designed  to  process  the  radar 
information,  resulting  in  the  radar  displays  currently  used 
by  air  traffic  controllers  (Office  of  Public  Affairs, 

1987a: 1) . 

The  Host  Computer  System  developed  by  IBM  was  one  of 
the  first  installments  of  the  FAA  multi-billion  dollar  plan 
to  modernize  the  National  Airspace  System  (NAS) .  Host 
serves  as  the  backbone  processor  for  the  new  Initial  Sector 
Suite  System  until  replacement  by  Advanced  Automation 
Processors  in  the  mid-1990s  (Garwood,  1988b}.  IBM  won  this 
197  million  dollar  contract  in  July  1985. 

Host  replaced  the  20-year-old  IBM  9020  computers  that 
were  located  in  the  nation's  20  Air  Route  Traffic  Control 
Centers  (ARTCC) .  The  new  IBM  mainframe  computers  were  ten 
times  faster  and  had  five  times  the  storage  capacity  of  the 
old  IBM  9020  computers.  The  replacement  involved  minimal 
changes  to  existing  National  Airspace  System  (NAS)  software 
(Office  of  Public  Affairs,  1987a: 1).  Total  NAS  software  had 
more  than  t«ro  million  lines  of  code,  and  Host  had  130,000 
lines  of  new  or  modified  NAS  code  (Office  of  Public  Affairs. 
1987b: 1).  According  to  FAA  Administrator  Donald  D.  Engen, 
'[Host]  will  allow  the  air  traffic  control  system  to  keep 
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pac«  with  projected  traffic  growth  over  the  next  decade  and 
accommodate  the  introduction  of  new  automation  functions 
that  will  both  enhance  safety  and  increase  controller 
productivity'  (Office  of  Public  Affairs,  1987a: 1).  Host 
features  included  increased  capacity,  improved  reliability, 
faster  response  time,  and  greater  on-line  availability 
(Office  of  Public  Affairs,  1987b: 1). 

Host  Success 

According  to  James  O.  Cain,  Deputy  Director  of  the 
Advanced  Automation  Program  Office,  ‘the  Host  program  has 
been  one  of  the  most  successful  technical  efforts  in  FAA 
history.  Beginning  with  the  first  delivery  to  the  Seattle 
center  in  November  1986,  the  FAA-IBM  team  has  met  or 
surpassed  every  major  program  milestone*  (Office  of  Public 
Affairs,  1987c:2). 

Gail  Garwood,  Host  Computer  System  Planning  and 
Implementation  Specialist,  presented  the  following  items 
used  as  proof  that  Host  was  successful  (Garwopd,  1988a): 

1 .  Every  ARTCC  accepted  the  system.  This  was  a  direct 
measure  of  user  confidence.  In  the  past  some  systems  were 
sent  back.  With  Host,  sites  requested  permission  to  drop 
some  preliminary  tests  and  accept  early. 

2.  Every  ARTCC  used  the  system  after  acceptance.  It  is 
up  to  the  site  manager  whether  to  use  a  new  system  or  not. 
There  are  currently  some  devices  on  ARTCCs  that  site 
managers  refuse  to  use. 

3.  Host  met,  or  was  ahead  of,  the  implementation 
schedule  in  all  areas. 
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4.  All  reliability,  maintainability  and  availability 
(RMA)  standards  and  technical  performance  measurements  (TPM) 
were  met  or  exceeded.  This  was  documented  in  test  results. 

5.  ARTCCs  requested  that  the  transition  switches  be 
disconnected  60  days  early. 

6.  Host  came  in  under  budget. 

According  to  James  Q.  Cain.  Deputy  Director  of  the 
Advanced  Automation  Program  Office,  *Much  of  the  credit  for 
this  [Host  success]  goes  to  the  FAA  and  contractor  personnel 
in  the  field.  They  have  been  committed  from  the  start  to 
making  this  program  work  and  have  done  an  outstanding  job' 
(Office  of  Public  Affairs,  1987c:2). 

Summarizing  the  beliefs  of  the  Host  planning  and 
control  special ists.,  Milton  Garwood  states  that,  'The  Host 
Computer  Program  has  been  one  of  the  most  successful 
technical  efforts  in  FAA  history'  because  of  the  total 
dedication  of  all  FAA  and  contractor  personnel  (Garwood, 
1988b).  A  spirit  of  'team  work*  has  prevailed  at  all 
levels . 

In  an  FAA  memorandum  Carlo  Yulo,  the  Operational  Test  & 
Evaluation  Division  manager,  wrote,  'Much  of  the  success  in 
the  first  Host  computer  system  implementation  effort  is 
attributable  to  the  commitment  and  esprit  de  corps  of  the 
people  involved...'  ( Yulo ,. 1988 : 1 ) . 

Richard  Marek ,  Program  Manager,  Host  Implementation 
Program,  stated  that  Host  was  the  most  successful  rehosting 
in  government  (Marek,  1987:1).  According  to  Mr.  Marek,  Host 
met  or  exceeded  every  requirement  because  of  intensive 
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monitoring  and  control  actions.  He  also  attributed  Host 
success  to  the  gathering  and  subsequent  use  of  feedback  from 
all  program  levels.  To  emphasize  the  Innovation  of  the  Host 
contract  administration  Mr.  Marek  listed  the  following  'Host 
firsts,'  elements  that  had  never  been  used  in  an  FAA  program 
before : 

1.  Extensive  field  involvement.  Four  million  dollars 
was  spent  to  involve  the  user. 

2.  Extensive  unstructured  testing  to  ensure  that  the 
system  would  not  do  anything  unexpected. 

3.  Use  of  an  award  fee  type  contract  to  motivate  the 
contractor.  IBM  received  80  percent  of  the  payment  on  ARTCC 
acceptance  as  long  as  there  were  no  type  I  errors.  The 
other  20  percent  was  awarded  when  IBM  achieved  RMA  and  TPMs 
above  contract  level. 

4.  All  line  requirements  in  the  contract  were 
identified  and  specific  responsibility  was  assigned  to  an 
individual  to  ensure  that  requirement  was  achieved.  Both 
the  contractor  and  the  responsible  FAA  individual  signed 
their  Initials  beside  each  requirement  after  it  was 
validated.  Between  500  and  1,000  requirements  were 
identified  and  individually  validated. 

5.  All  problems,  regardless  of  who  identified  them  or 
which  test  they  were  identified  during,  were  recorded  and 
visible  in  the  INFO  problem  system. 

These  Host  'firsts'  will  be  discussed  and  analyzed  with  the 
other  elements  perceived  by  the  Host  program  management  team 
to  contribute  to  Host  success  in  the  following  sections. 

The  sections  correspond  to  the  basic  concepts  underlying  the 
elements  identified  by  the  Host  program  management  team. 
Previous  study  of  classical  management  theory  and 
organizational  dynamics,  and  review  of  the  recommendations 
of  contemporary  acquisition  management  research  were  used  to 


extract  these  underlying  concepts.  The  six  concepts 
extracted  are:  1)  contract,  2)  teamwork,  3)  participative 
leadership,  4)  focus,  5)  contractor  and  system  monitoring, 
and  6)  testing  and  training. 

Contract 

The  Host  program  management  team  members  believed  that 
the  first  step  to  Host  acquisition  success  was  to  establish 
an  environment  conducive  to  program  management  through 
elements  in  the  contract  and  statement  of  work.  Mike  Rymond 
stated  that  the  entire  award  fee  process  was  a  key  to  Host 
success,  and  that  the  award  fee  was  written  to  ensure  that 
the  contractor  emphasized  what  the  program  office  believed 
to  be  important.  Linda  Strand,  the  Contracting  Officer, 
expanded  on  the  need  to  use  contract  elements  for 
motivation.  She  noted  that  one  of  the  reasons  that  Host  was 
so  successful  was  because  the  contract  was  tailored  to  the 
different  elements  being  procured.  The  other  major  reason 
for  Host  success,  according  to  Linda,  was  the  joint  use  of 
an  incentive  fee  and  an  award  fee  in  the  contract.  She 
emphasized  that  one  fee  should  not  be  used  without  the 
other.  The  combination  of  the  award  fee  with  the  incentive 
fee  locked  the  contractor  in  and  motivated  the  contractor  to 
meet  both  schedule  and  cost  objectives.  To  use  an  award  fee 
without  an  incentive  fee  could  cause  schedule  slippage,  and 
to  use  an  award  fee  without  an  incentive  fee  could  cause 
cost  overruns. 
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The  Host  computer  system  was  implemented  at  twenty 
different  sites.  One  of  the  elements  credited  with  Host 
success  was  the  ‘waterfall*  implementation  schedule  that  was 
used.  Carroll  Workman  was  not  alone  in  his  belief  that 
passing  the  lessons  learned  from  each  site  implementation  to 
subsequent  sites  was  critical.  As  Host  implementation 
progressed  the  number  of  problems  significantly  decreased, 
reflecting  the  usefulness  of  sequential  implementation. 

The  Host  program  management  team  believed  that  a 
program  schedule  objective  could  not  be  achieved  unless  the 
program  started  with  a  realistic  schedule.  Carroll  Workman 
emphasized  the  amount  of  work  put  into  establishing  a 
realistic  schedule  for  Host.  The  schedule  was  developed  by 
working  backwards  from  the  twenty  projected  delivery  dates, 
and  included  a  degree  of  slack  to  cover  program  risk.  He 
attributed  much  of  Host  achieving  success  to  the  use  of  a 
realistic  schedule. 

Diane  Ravenscroft  noted  that  one  of  the  ‘Host  firsts* 
was  to  include  money  in  the  program  budget  to  support 
participative  requirements.  She  viewed  this  as  a  major 
reason  for  Host  success  because  without  those  funds  the 
program  office  would  have  been  unable  to  involve  the  user  to 
the  extent  necessary. 

The  program  office  worked  very  closely  with  the  user  to 
develop  the  Host  requirements.  This  was  an  area  of  concern 
because  lack  of  success  in  previous  programs  had  been 
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partially  attributed  to  user  changes  in  requirements.  After 
being  closely  involved  in  developing  the  requirements  the 
user  did  not  need  to  make  subsequent  changes .  As  Don  Leabo 
stated,  'first  you  have  to  force  people  to  organize  what 
they  want,  then  you  can  provide  it*  (Leabo,  1988). 

Preston  Martin  and  Gail  Garwood  were  Just  two  members 
of  the  program  management  team  who  perceived  that  the 
contract  clause  tying  problem  resolution  to  the  contractor's 
payment  schedule  was  a  key  contributor  to  Host  success.  By 
joinly  classifying  and  prioritizing  problems,  the  contractor 
was  able  to  maximize  the  benefit  from  scarce  manpower 
resources.  This  contract  element  also  motivated  the 
contractor  to  resolve  problems  as  soon  as  possible  because 
lack  of  timely  resolution  Impacted  subsequent  deliveries  and 
payment . 

The  contract  concept  management  procedures  and 
organizational  elements  perceived  by  the  Host  program 
management  team  to  be  vital  to  Host  success  are  summarized 
in  Table  4.1.  These  elements  partially  answer  investigative 
question  3b. 

Teamwork 

The  following  statement  by  Arthur  Simolunas, 

Contracting  Officer  Technical  Representative,  that  the 
biggest  thing  that  contributed  to  Host  success  was  the 
teamwork  summarized  the  Host  program  management  team's 
perception  of  their  joint  role  with  the  contractor.  A  term 
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Table  4.1 


Concept:  Contract 

Summary  of  The  Elements  Perceived  By 
The  Host  Program  Management  Team 
To  Contribute  To  Host  Success 

»  Award  Fee  and  Incentive  Fee  Used  Jointly  (CPIF/AF) 

*  Waterfall  Implementation  Plan 

*  Realistic  Schedule  Providing  Some  Slack 

«  Funds  Budgeted  to  Support  Participative  Requirements 

*  Detailed  Requirements  Generated  by  the  User 

»  Problem  Resolution  Impacts  Contractor  Payment  Schedule 


frequently  used  by  the  Host  team  members  was  ‘unified 
front.’  This  term  symbolized  all  of  the  teamwork  elements 
perceived  to  contribute  to  Host  success.  The  Host  ‘unified 
front’  was  accomplished  through  what  was  described  as  a 
painful  process  of  frequent  meetings  to  resolve  differences 
in  program  perceptions  and  goals.  The  result  of  these 
meetings  was  a  determination  of  what  the  Joint  position 
would  be,  and  the  full  support  by  both  FAA  and  IBM  team 
members  of  that  position.  These  comments  about  concern  for 
teamwork  and  unity  were  corroborated  by  the  data  collected 
from  the  direct  observation  of  Host  events.  Refer  to 
Appendix  B:  Host  Computer  Program,  Events  Observed,  for  a 


listing  and  description  of  the  events  observed.  During  the 
these  events,  both  government  and  contractor  representatives 
showed  an  obvious  desire  to  cooperate.  Within  the  FAA  team 
the  substantial  effort  to  minimize  the  long-standing 
conflict  between  Air  Traffic  (AT)  and  Airway  Facilities  (AF) 
personnel  was  evident.  For  example,  two  regional 
representatives,  one  AT  and  one  AF,  traveled  to  the  site 
survey  together,  and  prepared  a  joint  site  visit  report  on 
their  return.  Normally  two  different  reports  reflecting  two 
different  views  would  have  been  filed. 

James  (JR)  Young,  Host  System  Implementation 
Specialist,  stated  that  in  the  beginning  no  one  knew  who  was 
responsible  for  what,  so  everyone  applied  common  sense  and 
asked  a  lot  of  questions.  He  admitted  that  early  in  the 
program  there  was  a  lot  of  reluctance  to  cooperate,  but 
added  that  a  cohesive  team  was  built  before  the  contract  was 
even  awarded.  JR  believed  that  involving  all  team  members 
in  planning  and  scheduling  helped  to  ensure  that  everyone 
was  going  in  the  same  direction,  and  to  ensure  that  all 
areas  were  covered.  Lateral  communication  was  perceived  to 
be  a  vital  element  in  facilitating  team  cohesiveness.  To  be 
effective,  team  members  had  to  be  given  the  flexibility  to 
go  searching  for  answers  informally  across  functional 
boundaries . 

James  Young  noted  that  top  management  had  confidence  in 
the  team’s  ability,  and  let  the  team  members  set  up  their 
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own  Informal  team  structure  and  accept  responsibilities 
instead  of  having  responsibilities  assigned.  The  team 
members  perceived  the  freedom  to  accept  responsibilities  and 
to  be  innovative  as  key  elements  in  achieving  Host  success. 

The  Host  concept  of  teamwork  also  extended  to  problem 
resolution.  In  the  beginning  of  the  contract  performance 
period  IBM  ran  into  software  development  problems.  Both  IBM 
and  FAA  program  team  members  believed  that  Host  would  not 
have  been  successful  if  their  efforts  had  not  been  pooled  to 
resolve  the  problems. 

The  teamwork  concept  management  procedures  and 
organizational  elements  perceived  by  the  Host  program 
management  team  to  be  vital  to  Host  success  are  summarized 
in  Table  4.2.  These  elements  provide  an  additional  piece  of 
the  answer  to  investigative  question  3b. 

Table  4.2 
Concept:  Teamwork 

Summary  of  The  Elements  Perceived  By 
The  Host  Program  Management  Team 
To  Contribute  To  Host  Success 

*  All  Constituents  Involved  in  Planning  and  Scheduling 

*  Lateral  Communication  Channels  Open 

*  Frequent  Meeting  Used  to  Resolve  Differences 
»  Responsibilities  Accepted,  Not  Assigned 

*  One  Face  to  Users  and  Management 

»  All  Functions  Involved  in  Problem  Resolution 
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Participative  Leadership 

The  perception  that  user  involvement  contributed 
directly  to  Host  success  permeated  every  interview.  As 
stated  by  Don  Leabo ,  'involving  users  and  operators  at  every 
step  of  Host  development  caused  some  problems,  but  the 
benefits  greatly  outweighed  those  problems'  (Leabo,  1988). 
Users  helped  to  make  design  tradeoffs,  aided  software 
debugging,  and  were  directly  Involved  in  testing.  User 
involvement  was  credited  with  facilitating  implementation. 

The  dedication  to  user  Involvement  was  reflected  in  the 
four  million  dollars  budgeted  and  funded  to  support 
participative  requirements.  This  dedication  was  reflected 
in  the  detail  of  information  provided  to  users,  and  the 
quantity  of  feedback  gathered  from  users.  The  users  were 
educated  about  Host  contract  administration,  provided  with 
points  of  contact  in  case  of  problems,  forwarned  of 
potential  problems  to  watch  for,  and  given  access  to  INFO, 
the  problem  database.  The  Host  program  management  team 
believed  that  keeping  users  informed  was  vital  to  Host 
success.  The  team  stated  that  requesting  feedback  from  the 
user  contributed  to  user  acceptance.  They  also  stated  that 
using  the  feedback  requested  was  critical  to  obtaining  user 
support.  Allowing  users  to  voice  disagreements  and  concerns 
was  perceived  to  be  an  important  element  in  preventing  minor 
details  from  halting  or  hindering  implementation. 
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Andrew  Bowman,  a  computer  technician  at  the 
Indianapolis  ARTCC  summarized  the  importance  user 
Involvement.  He  stated,  ‘People  are  not  worried  about  Host, 
they  are  eager  and  excited'  (Bowman,  1967).  When  asked  why 
he  believed  that  the  users  were  enthusiastic  he  replied, 
‘Because  they  were  briefed  from  the  beginning  on  the 
benefits  of  Host,  because  they  were  told  what  their 
responsibilities  would  be  and  how  they  would  fit  into  Host 
implementation  and  use,  and  because  everyone  was  trained  and 
confident  that  they  would  be  able  to  perform  their  job  when 
the  new  system  arrived'  (Bowman,  1987). 

The  participative  leadership  concept  management 
procedures  and  organizational  elements  perceived  by  the  Host 
program  management  team  to  be  vital  to  Host  success  are 
suimnarized  in  Table  4.3.  These  elements  provide  an 
additional  piece  of  the  answer  to  investigative  question  3b. 

Focus 

Don  Leabo  summarized  the  program  management  team’s 
perception  of  the  elements  of  configuration  management  when 
he  said  that  to  keep  changing  the  requirements  or  program 
scope  was  to  never  finish  the  program.  Michael  Perie  added 
that  Host  was  a  difficult  program,  but  that  staying  within 
the  original  program  scope,  and  ensuring  that  specifications 
were  met  resulted  in  program  success.  Mr.  Perie  emphasized, 
‘What  we  said  we  were  going  to  do  in  1982  is  what  we  focused 
on  doing,  and  finally  did'  (Perie,  1987).  Every  team 
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Table  4.3 


Concept:  Participative  Leadership 

Summary  of  The  Elements  Perceived  By 
The  Host  Program  Management  Team 
To  Contribute  To  Host  Success 

»  Site  Representatives  Involved  From  Start  to  Finish 
«  User  Input  Requested  In  All  Areas  and  All  Phases 
»  Site  Input  Actually  Used 

»  Users  Kept  Informed,  Frequent  Site  Briefings  and  Memos 
«  Problems  Solved  at  the  Lowest  Level  Possible 
»  Users  Allowed  to  Voice  Disagreements  and  Concerns 


member  commented  that  an  important  factor  in  achieving 
success  was  keeping  changes  to  a  minimum.  This  ment  that  no 
elements  determined  to  be  in  the  "nice  to  have"  category 
were  added.  The  team  members  emphasized  that  requirements 
stability  and  focus  on  program  scope  contributed  to  Host 
meeting  budget  and  schedule  objectives. 

Configuration  management  principles  were  extended  to 
managing  the  studies  performed  by  Host  support  contractors. 
Don  Leabo  stated  that  a  key  to  Host  success  was  focusing 
support  contractors  and  screening  all  proposed  studies.  He 
emphasized  that  an  independent  group  was  needed  to  assess 
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program  status,  but  that  only  those  studies  that  would 


eliminate  some  nonperformance  or  increase  performance  should 
be  allowed. 

The  focus  concept  management  procedures  and 
organizational  elements  perceived  by  the  Host  program 
management  team  to  be  vital  to  Host  success  are  summarized 
in  Table  4.4.  These  elements  provide  an  additional  piece  of 
the  answer  to  investigative  question  3b. 

Table  4.4 

Concept:  Focus 

Summary  of  The  Elements  Perceived  By 
The  Host  Program  Management  Team 
To  Contribute  To  Host  Success 

»  No  Bells,  Whistles,  or  Additional  Fixes  Were  Added 
»  Innovation  Restricted  to  the  Original  Program  Scope 
*  Studies  Performed  by  Support  Contractors  Limited 

Contractor  and  System  Monitoring 

The  Host  program  management  team  had  made  detailed 
plans  and  preparations  for  how  they  were  going  to  ensure 
that  the  system  received  met  all  of  the  specifications 
outlined  in  the  contract,  and  for  how  they  were  going  to 
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ensure  that  program  goals  were  met.  These  plans  and 
preparations  were  perceived  to  contribute  directly  to  Host 
success . 

Don  Leabo  stated  that  Host  met,  or  exceeded,  every 
requirement  because  each  requirement  was  assigned  to  a 
specific  FAA  individual  who  was  held  accountable  to  ensure 
that  the  requirement  was  met.  Mike  Rymond  explained  the 
concept.  He  stated  that  the  contract  requirements  were 
broken  into  like-groups,  an  appropriate  FAA  team  member  was 
put  in  charge  of  each  group,  and  the  program  schedule  was 
built  around  the  dates  that  each  group  of  requirements  was 
to  be  met.  He  added  that  the  FAA  team  member  was  held 
accountable  for  validating  that  the  group  of  requirements 
were  met,  and  was  required  to  initial  beside  each 
requirement  as  it  was  met. 

Another  part  of  the  Host  program  management  team’s 
planning  and  preparation  for  program  success,  was 
establishing  the  standards,  by  which  the  system  was  later 
judged,  up  front.  Program  management  team  members  also 
emphasized  the  importance  of  ensuring  that  the  standards 
were  achievable. 

A  management  information  system  developed  by  IBM, 
attributed  with  contributing  to  Host  success,  was  the  INFO 
system.  INFO  was  used  to  trace  and  record  all  of  the 
problems  identified  in  Host.  It  was  accessible  from  every 
FAA  site,  and  was  viewed  by  the  Host  program  management  team 
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as  invaluable.  Once  a  problem  was  entered  into  INFO  it 
could  not  be  eliminated,  only  categorized.  Problems  could 
only  be  categorized  as  resolved  by  the  individual  who 
entered  the  problem  originally.  This  procedure  ensured  that 
nothing  was  covered  up  or  hidden.  Some  of  the  benefits  of 
having  all  problems  visible  were;  1)  trends  could  be 
identified,  and  2)  resolution  efforts  were  not  duplicated. 
INFO  was  also  attributed  with  increasing  Host  reliability, 
and  helping  with  configuration  management. 

A  significant  event,  termed  the  six  month  slip, 
revealed  the  importance  of  monitoring  program  progress 
parallel  to  contractor  monitoring.  The  FAA  team  believed 
that  is  was  important  not  to  go  solely  by  the  contractor’s 
progress  reports  because  they  would  slant  the  reports  to 
their  best  benefit.  Without  the  parallel  monitoring,  both 
Bennie  Sanford  and  Don  Leabo  believed  that  what  ended  up  as 
a  six  month  delay  would  have  been  an  eighteen  month  delay. 

The  contractor  and  system  monitoring  concept  management 
procedures  and  organizational  elements  perceived  by  the  Host 
program  management  team  to  be  vital  to  Host  success  are 
summarized  in  Table  4.5.  These  elements  provide  a  piece  of 
the  answer  to  investigative  question  3b. 

Testing  and  Training 

The  Host  program  management  team  members  carefully 
evaluated  the  importance  of  testing  and  training  during 
initial  program  planning  and  scheduling.  Both  were 
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Table  4.5 


Concept:  Contractor  and  System  Monitoring 

Summary  of  The  Elements  Perceived  By 
The  Host  Program  Management  Team 
To  Contribute  To  Host  Success 

»  Actual  Demonstration  of  Requirement  Before  Sign-off 

*  Technical  Performance  Measures  Established  Up  Front 

*  Repairabi 1 i ty ,  Maintainability,  and  Achievabi 1 i ty 
Standards  Established  Up  Front,  and  Achievable 

»  All  Problems  Categorized  and  Visible  in  INFO  Database 

*  Numerous  Indicators  of  Program  Progress  Used 


determined  to  be  significant  factors  in  whether  a  program 
would  be  successful  or  not.  Don  Leabo  stated  that  testing 
must  be  conducted  as  far  up  front  as  possible.  He  added 
that  early  testing  was  expensive,  but  countered  that  it  had 
to  be  accomplished  whether  it  was  done  early  or  not.  Don 
also  stated  that  if  teting  was  done  early  there  would  be 
more  lead  time  for  problem  resolution. 

One  of  the  Host  'firsts*  was  to  conduct  unstructured 
testing.  This  testing  was  conducted  by  users  without  a 
scripted  scenario.  The  unstructured  tests  revealed  hundreds 
of  problems  above  and  beyond  those  already  identified  by  the 
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structured  testing.  Don  Leabo  pointed  out  that  unstructured 
testing  was  necessary  because  structured  testing  and  related 
fixes  to  problems  identified  could  be  manipulated,  but  with 
unstructured  testing  they  could  not. 

Involving  the  user  in  testing  was  perceived  to  be  vital 
to  the  quick  acceptance  of  the  Host  system.  This 
involvement  also  was  credited  with  gaving  sites  an  emotional 
stake  in  implementation,  and  with  improving  Host 
rel iabi 1 1 ty . 

The  Host  training  requirements  were  developed  with  user 
input,  and  follow  up  was  conducted  to  assess  effectiveness. 
User  training  was  believed  to  directly  affect  user 
acceptance.  Milt  Qarwood  explained  the  importance  of  Host 
training  before  system  delivery.  He  noted  that  Host  would 
change  the  clerical  role  of  the  computer  technicians  to  a 
role  directly  interfacing  with  the  system.  To  increase  user 
confidence  and  to  ensure  that  the  site  would  be  prepared 
before  Host  delivery  the  operators  were  sent  to  the  FAA 
Academy  for  six  weeks  of  training.  The  computer  operators 
were  also  involved  in  system  testing. 

After  Host  delivery,  site  specific  testing  was 
conducted  to  fine-tune  the  system,  and  to  ensure  that  there 
were  no  major  problems.  This  testing  also  uncovered  lessons 
learned  that  were  passed  on  to  other  sites. 

The  testing  and  training  concept  management  procedures 
and  organizational  elements  perceived  by  the  Host  program 
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management  team  to  be  vital  to  Host  success  are  summarized 
in  Table  4.6.  These  elements  provide  the  final  piece  of  the 
answer  to  investigative  question  3b. 


Table  4.6 

Concept:  Testing  and  Training 

Summary  of  The  Elements  Perceived  By 
The  Host  Program  Management  Team 
To  Contribute  To  Host  Success 

*  Testing  as  Early  in  Development  as  Possible 

»  Both  Structured  and  Unstructured  Testing  Conducted 

*  User  Involved  in  Planning  and  Performing  Test  Scenarios 
»  Site  Personnel  Trained  Before  System  Delivery 

»  After  Delivery,  Detailed  Site  Specific  Testing  Conducted 


« 


■ 
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Comparison  With  Recommendations  In  Previous  Research 
The  literature  review  conducted  in  Chapter  II 
identified  eight  elements  recommended  in  previous  research 
to  improve  the  DOD  acquisition  process.  The  previous 
sections  of  this  chapter  have  cumulatively  answered 
investigative  question  3b  by  reporting  the  elements  that 
were  perceived  by  the  Host  program  management  team  to  be 


I 


J 
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vital  to  Host  success.  The  next  step  is  to  evaluate  the 
elements  identified  by  the  Host  program  management  team  with 
the  elements  recommended  in  Chapter  II.  The  following 
paragraphs  will  reiterate  the  recommendations  from  previous 
research,  and  compare  them  to  the  findings  from  the 
exploratory  case  study  of  Host.  The  conclusions  drawn  from 
these  paragraphs  will  be  used  to  answer  investigative 
question  4. 

The  need  for  program  manager  authority  and  flexibility 
was  identified  in  all  of  the  studies  reviewed.  The  Host 
program  manager  was  given  sufficient  authority  and 
flexibility  to  make  tradeoff  decisions  as  necessary,  and 
delegated  substantial  responsibility  and  authority  to  the 
team  members.  This  delegation  permitted  the  Host  program 
management  team  to  be  innovative  and  flexible.  The  elements 
in  the  Host  teamwork  concept  reflect  the  need  for  this 
authority  and  freedom. 

The  need  for  program  stability  was  recognized  both  by 
previous  research  and  by  the  Host  program  management  team. 
The  elements  in  the  Host  focus  concept  reflect  the  need  for 
requirements  stability. 

Previous  research  also  noted  that  a  small,  high  quality 
staff  was  a  common  feature  of  successful  programs.  The  Host 
program  management  team  members  did  not  perceive  this  to  be 
a  factor  in  achieving  Host  program  success.  Evaluation  of 
the  efforts  expended  by  the  team  members  did  indicate  that 
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the  group  was  small  and  of  high  quality.  This 
recommendation  is  not  part  of  the  teamwork  concept  elements, 
but  it  was  a  part  of  the  Host  program. 

A  program  office  team  atmosphere,  supported  by  more 
specific  recommendations,  was  identified  by  previous 
research  as  an  important  factor  in  program  success.  The 
Host  program  management  team  also  perceived  teamwork  to  be  a 
critical  factor  in  the  success  of  their  program. 

The  need  to  communicate  with  users  was  identified  as  a 
factor  common  to  successful  programs,  and  was  a 
recommendation  for  Improving  the  DOD  acquisition  process. 

The  elements  of  the  Host  participative  leadership  concept 
also  reflect  this  need. 

Prototyping  and  testing  were  recommended  by  previous 
research  as  methods  for  uncovering  problem  areas  as  well  as 
methods  to  improve  cost  and  schedule  estimates.  The  Host 
program  did  not  involve  prototyping,  so  that  section  of  the 
recommendation  is  not  a  factor  in  this  evaluation.  The  need 
for  testing  was  identified,  and  an  entire  group  of  elements 
within  the  context  of  testing  were  perceived  by  the  Host 
program  management  team  to  contribute  to  Host  success. 

A  second  area  of  stability  addressed  by  previous 
research  was  the  recommendation  that  program  managers 
strictly  adhere  to  the  established  schedule.  The  Host 
program  management  team  addressed  scheduling,  but  their 
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focus  was  on  the  need  for  a  realistic  schedule  to  begin 
with . 

The  final  reconunendation  listed  in  Chapter  II  is  the 
need  for  a  teamwork  relationship  between  the  government  and 
the  contractor.  The  Host  program  management  team  perceived 
this  to  be  a  critical  area.  Several  elements  in  the  context 
of  teamwork  were  perceived  by  the  Host  team  to  contribute  to 
Host  success. 

In  summary,  the  answer  to  investigative  question  4  is 
that  the  elements  perceived  by  the  Host  program  management 
team  to  contribute  to  Host  success  are  in  agreement  with  the 
recommendations  of  previous  research.  The  first  research 
objective  has  been  achieved.  The  next  chapter  presents  the 
findings  and  analysis  to  achieve  the  second  resea-*ch 
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V.  Expert  Survey  Findiims  and  Analysis 

Chapter  Overview 

This  chapter  presents  a  discussion  of  the  data 
collected  during  interviews  with  DOD  acquisition  experts. 

The  expert  survey  findings  and  analysis  are  divided  into 
seven  sections.  First,  the  research  question  and  supporting 
investigative  questions  to  be  answered  in  this  chapter  are 
restated.  The  second  section  describes  the  effectiveness  of 
the  data  collection  method.  The  next  three  sections  discuss 
the  responses  received  to  the  interview  questions ,  analyze 
those  responses,  and  present  the  findings.  The  findings 
presented  in  these  three  sections  contain  the  answers  to 
investigative  question  5.  Several  of  the  respondents 
identified  elements  that  they  believed  were  necessary  for 
program  success  and  were  missing  from  the  hypothesized 
management  strategy.  These  recommendations  are  presented 
and  evaluated  in  tne  next  section.  In  the  final  section  the 
management  strategy  that  accomplishes  the  second  research 
objective  is  outlined. 

Research  Objective  and  Supporting  Investigative  Questions 

DOD  acquisition  experts  analyzed  the  elements 
identified  during  the  exploration  of  the  Host  computer 
system.  This  analysis  provided  the  data  to  achieve  the 
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second  research  objective.  That  analysis  and  the  resulting 
management  strategy  are  presented  in  this  chapter. 

Research  Objective  II.  To  develop  an  acquisition 
atanageoMnt  strategy  based  on  the  elements  for  success 
identified  that  is  applicable  to  DOD  program  management. 

Investigative  Question  5.  Which  of  the  elements  for 
success  resulting  from  the  analysis  of  the  successful 
program  and  previous  research  form  the  management  strategy 
that  will  potentially  increase  the  DOD  acquisition  success 
rate?  This  investigative  question  is  answered  through  the 
following  seven  sub-questions: 

a.  Which  of  these  elements  for  success  do  DOD 
acquisition  experts  identify  as  ideas  new  to  DOD? 

b.  Which  of  these  elements  for  success  do  DOD 
acquisition  experts  identify  as  necessary  for  program 
success? 

c.  Of  the  elements  identified  by  DOD  acquisition 
experts  as  necessary  for  program  success,  which  are 
currently  used  and  which  are  potentially  applicable  to  the 
DOD  acquisition  process? 

d.  What  must  be  done  before  the  elements  for  success 
potentially  applicable  to  the  DOD  acquisition  process  can  be 
implemented? 

e.  For  those  elements  identified  by  DOD  acquisition 
experts  as  not  potentially  applicable  to  the  DOD  acquisition 
process,  why  were  they  so  identified? 

f.  For  those  elements  identified  by  DOD  acquisition 
experts  as  currently  used  or  potentially  applicable  what  is 
the  range  of  applicability? 


Data  Collection 


A  primary  goal  of  the  interviewer  was  to  motivate  the 
respondent  to  accept  his/her  role  in  the  survey  process  and 
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to  fulfill  the  requirements  of  that  role  (Emory,  1985:161). 
All  of  the  respondents  were  interested  in  the  research  area 
and  were  enthusiastic  about  participating.  The  willingness 
of  the  respondents  to  accept  their  roles  in  the  survey 
process  was  reflected  in  the  asiount  of  time  that  they 
willingly  gave  to  the  interview.  A  time  block  of  30  to  45 
minutes  was  scheduled  for  each  interview.  The  shortest 
interview  lasted  1.0  hour,  the  longest  lasted  3.5  hours,  and 
the  average  was  1.5  hours.  Another  indication  of  respondent 
motivation  was  the  volume  of  unsolicited  comments  and 
documents.  Five  of  the  eleven  respondents  provided  copies 
of  new  initiatives,  regulations,  and  document  drafts  to 
augment  their  comments.  All  but  two  of  the  respondents 
addressed  issues  and  made  comments  above  and  beyond  those 
outlined  in  the  interview  protocol. 

Each  respondent  was  asked  the  same  general  questions 
for  each  element  in  the  hypothesized  management  strategy. 

The  respondent  was  asked  to  clarify  points  or  to  provide 
additional  information  when  needed.  Appendix  E:  POD  Expert 
Interview.  Interview  Session  Organization,  lists  the 
questions  asked  of  each  respondent.  The  data  collected  from 
these  questions  was  used  to  answer  investigative  question  5. 

The  respondents  addressed  only  the  categories  and 
elements  in  which  they  felt  sufficiently  knowledgeable. 
Expert  survey  protocol  allows  respondents  to  discuss  the 
interview  questions  with  few  constraints  and  to  limit 
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comments  to  those  areas  which  they  determine  that  they  are 
qualified  to  address  (Zikmund,  1985:103).  In  addition,  the 
respondents  primarily  commented  on  the  elements  that  they 
either  advocated  or  opposed.  The  elements  that  respondents 
believed  were  obviously  necessary  or  regularly  applied 
received  few  comments.  At  the  conclusion  of  the  expert 
survey,  all  of  the  elements  had  been  evaluated  and  the 
interview  questions  had  been  answered. 

Each  respondent  received  a  thank  you  letter  and  a 
paraphrase  of  his/her  comments  for  review  and  approval.  The 
instructions  stated  that  if  a  response  was  not  received, 
then  the  paraphrase  would  be  assumed  to  be  approved  as  it 
was.  Two  months  later,  nine  responses  had  been  received  and 
all  of  the  paraphrases  were  finalized.  Appendix  H  contains 
an  approved  interview  paraphrase  for  each  respondent  and  is 
organized  in  alphabetical  order  by  respondent. 

The  hypothesized  management  strategy  which  resulted 
from  the  exploration  of  the  successful  Host  computer  system 
acquisition  is  outlined  in  Table  5.1,  Reasons  For  Host 
Success .  The  expert  respondents  were  given  a  similar 
outline  to  facilitate  their  evaluation.  Table  5.1  reflects 
the  corresponding  code  for  each  element.  This  code  is  used 
throughout  the  chapter,  in  both  text  and  tables,  to 
reference  specific  elements. 
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Table  5. 1 


Reasons  For  Host  Success: 
Elements  Listed  With  Reference  Code 


Code  Element 

Contract : 

Cl.  Award  Fee  and  Incentive  Fee  used  jointly  (CPIF/AF) 

C2.  Waterfall  implementation  plan 

C3.  Realistic  schedule  providing  some  slack 

C4.  Funds  budgeted  to  support  participative  requirements 

C5.  Detailed  requirements  generated  by  the  user 

C6 .  Problem  resolution  impacts  contractor  payment  schedule 

Teamwork : 

Tl.  All  constituents  involved  in  planning  and  scheduling 

T2 .  Lateral  communication  channels  open 

T3.  Frequent  meetings  used  to  resolve  differences 

T4 .  Responsibilities  accepted,  not  assigned 

T5.  One  face  to  users  and  management 

T6.  All  functions  involved  in  problem  resolution 

Participative  Leadership: 

PI.  Site  representatives  involved  from  start  to  finish 
P2.  User  input  requested  in  all  areas  and  all  phases 
P3.  Site  input  actually  used 

P4 .  Users  kept  informed,  frequent  site  briefings  and  memos 

P5 .  Problems  solved  at  the  lowest  level  possible 

P6.  Users  allowed  to  voice  disagreements  and  concerns 

Focus : 

FI.  No  bells,  whistles,  or  additional  fixes  were  added 
F2 .  Innovation  restricted  to  the  original  program  scope 
F3 .  Studies  performed  by  support  contractors  limited 

Contractor  and  System  Monitoring: 

51.  Actual  demonstration  of  requirement  before  sign-off 

52.  Technical  Performance  Measures  established  up  front 

53.  RMA  established  up  front,  and  achievable 

54.  All  problems  categorized  and  visible  in  INFO  database 

55.  Numerous  indicators  of  program  progress  used 

Testing  and  Training: 

A1 .  Testing  as  early  in  development  as  possible 

A2 .  Both  structured  and  unstructured  testing  conducted 

A3.  User  involved  in  planning  and  performing  test  scenarios 

A4.  Site  personnel  trained  before  system  delivery 

AS.  After  delivery  detailed  site  specific  testing  conducted 
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Elenwints  Identified  as  Mew  Concepta 

Each  expert  respondent  was  asked  to  identify  the 
hypothesized  management  strategy  elements  that  contained  new 
or  unique  program  management  concepts.  This  information  was 
used  to  answer  investigative  question  5a.  Lt  Col  Paul 
Huegel  suBimarized  the  group  consensus  when  he  stated, 
'Everything  here  has  been  thought  of  long  before*  (Huegel, 
1988)  . 

One  element  was  identified  by  one  respondent  as  a 
potentially  new  concept.  Element  F3.,  Studies  performed  by 
support  contractors  limited,  was  identified  by  Col  Ralph 
Tourino  as  an  interesting  catch,  something  that  he  had  never 
seen  written  down  before.  To  answer  investigative  question 
5a,  it  was  found  that  none  of  the  elements  composing  the 
hypothesized  management  strategy  were  concepts  new  to  DOD. 

Element  Necessity 

This  section  presents  the  range  and  general  sentiment 
of  the  responses  received  to  the  second  interview  question. 
This  Information  was  used  to  answer  investigative  question 
5b.  The  second  interview  question  required  the  respondents 
to  evaluate  each  element  and  to  determine  if  that  element 
was  necessary  for  an  acquisition  program  to  be  successful. 
During  the  evaluation  the  respondents  identified  several 
elements  that  would  require  revision  before  being  considered 
to  be  necessary.  The  recommendations  are  presented  with  the 
analysis  of  the  appropriate  element.  For  the  elements 
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determined  by  general  consenaua  to  be  not  necessary,  the 
reasons  for  the  determination  are  presented.  The 
presentation  and  analysis  of  respondent  comments  are  divided 
into  six  sections.  The  sections  correspond  to  the  six 
categories  of  the  hypothesized  management  strategy. 

Tables  5.2  through  5.7  are  used  to  summarize  the 
responses  received  to  the  interview  question,  ‘Is  this 
element  necessary  for  an  acquisition  program  to  be 
successful?'  A  table  is  devoted  to  each  of  the  six 
categories.  Each  element  composing  the  hypothesized 
management  strategy  is  listed  by  its  reference  code  in  the 
table  corresponding  to  the  appropriate  category.  Table  5.1 
can  be  used  to  cross-reference  each  code  to  the  matching 
element . 

In  Table  5.2  through  5.7,  the  qualitative  comments  of 
each  respondent  were  quantified  into  the  appropriate  columns 
for  each  element.  The  No  Comment  column  was  selected  if  the 
respondent  did  not  address  the  element,  either  specifically 
or  indirectly,  during  the  interview.  The  Not  Necessary 
column  was  selected  if  the  respondent  specifically  mentioned 
that  the  element  was  not  necessary  for  program  success.  The 
Necessary  After  Revision  column  was  selected  when  the 
respondent  recommended  major  revisions  to  the  element,  and 
commented  that  the  modifications  would  make  the  element 
necessary.  The  Necessary  column  was  selected  both  when  the 
respondent  specifically  stated  that  the  element  was 
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n«c«asary,  and  when  the  respondent  indicated  approval  of  the 
element . 

The  numbers  listed  under  each  response  column  indicate 
the  percentage  of  respondents  whose  comments  about  the 
corresponding  element  fit  the  described  response.  The 
percentages  were  calculated  based  on  a  total  of  eleven 
respondents.  The  percentages  for  each  element  sum  to 
approximately  100  percent.  The  dashes  indicate  that  there 
were  no  responses  concerning  the  element  of  interest  that 
fit  that  described  response. 

Contract ■  This  section  discusses  and  analyzes  the 
responses  to  the  second  interview  question  for  the  elements 
listed  under  the  Contract  category.  The  section  is  divided 
into  seven  subsections.  A  subsection  is  devoted  to  each  of 
the  six  elements  in  this  category.  The  final  subsection 
will  summarize  the  results.  Table  5.2  summarizes  the 
findings  concerning  element  necessity  that  are  applicable  to 
the  Contract  category. 

Award  Fee  and  Incentive  Fee  Used  Jointly 
(CPIF/AF) .  All  of  the  respondents  addressing  element  Cl 
believed  that  it  was  necessary  for  program  success.  Ms. 
Darleen  Druyun  summarized  the  necessity  of  this  element,  'It 
is  important  to  match  contract  type  to  inherent  program 
risk.  If  this  is  not  done,  then  the  contract  is  put  into  a 
mission  impossible  situation*  (Druyun,  1968).  Mr.  Daniel 
Rak  emphasized  the  importance  of  element  Cl  when  he 
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Table  5.2 


Category:  Contract 


Summary  of  Responses  To  The  Interview  Question: 
Is  This  Element  Eecessary  For  An 
Acquisition  Program  To  Be  Successful? 

(in  percentages,  eleven  respondents) 


Code 


No 

Comment 

% 


Not 

Necessary 

X 


Necessary 
Af  ter 
Revision 
X 


Necessary 

X 


commented  that  maximum  latitude  in  management  and 
flexibility  could  be  attained  simply  by  varying  the  share 
ratio  between  incentive  fees  and  award  fees  (Kak,  1986). 
Based  on  unanimous  approval,  element  Cl  was  found  to  be 
necessary  for  program  success. 

Waterfall  Implementation  Plan.  Six  respondents 
addressed  element  C2.  They  unanimously  agreed  that  element 
C2  was  necessary  for  program  success.  The  group  consensus 
was  that  a  waterfall  implementation  plan  was  an  obvious 
requirement.  This  element  was  found  to  be  necessary  for 
program  success. 

Realistic  Schedule  Providing  Some  Slack.  For 
element  C3 ,  Mr.  Thomas  Christie’s  comments  epitomized  the 
opinions  of  the  eight  respondents  who  addressed  this 
element.  He  stated  that  a  realistic  schedule  providing  some 
slack  was  vital  to  program  success.  This  element  was  found 
to  be  necessary  for  program  success. 

Funds  Budffeted  To  Support  Participative 
Requirements .  There  were  a  wide  range  of  responses  to 
element  C4 .  Ten  out  of  eleven  respondents  addressed  this 
issue.  All  ten  agreed  that  it  was  necessary  to  include  the 
funding  requirements  to  support  user  participation  in  some 
budget  estimate.  The  disparity  of  opinion  was  whether  to 
include  the  participative  requirement  in  the  program  budget 
or  to  include  it  in  the  user’s  budget. 
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Tvfo  respondents  stated  that  element  C4  was  not 
necessary.  Their  opinions  were  capsulized  in  the  following 
statements  by  Joseph  Drelicharz,  *It  is  neither  wrong  nor 
bad  for  users  to  fund  their  own  TDYs  to  support  the  project. 
The  user’s  willingness  to  do  this  is  a  good  measure  of  user 
desire  and  interest*  (Drelicharz,  1988). 

Five  respondents  believed  that  it  was  not  important 
whether  the  user’s  funds  or  the  program’s  funds  were  used  to 
support  the  requirements.  Their  concern  was  that  the 
requirements  were  included  somewhere. 

Three  respondents  believed  that  element  C4  was  a  good 
idea,  and  worth  looking  into.  Lt  Col  Paul  Huegel  summarized 
the  opinions  of  the  other  t«ro  respondents  when  he  stated, 
'The  idea  here  is  good.  This  is  definitely  an  area  that 
requires  attention*  (Huegel,  1988).  Colonel  Huegel 
emphasized  the  importance  of  this  element  in  the  following 
statement.  *Qeneral  Randolph,  Commander  of  Air  Force  Systems 
Command  (AFSC) ,  wants  AFSC  people  to  OMet  with  the  user,  to 
keep  close  to  the  user,  and  to  know  the  user's  needs--but 
the  TDY  funds  are  not  there  to  do  it*  (Huegel,  1988). 

The  majority  of  the  respondents  recommended  that 
element  C4  be  revised.  They  believed  that  revision  of  this 
element  would  make  it  necessary  for  program  success.  It  was 
found  that  element  C4  required  revision.  In  addition,  it 
was  recommended  that  DOD  look  into  the  possibility  of 
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including  funding  for  participative  requirements  in  the 
program  budget. 

Detailed  Requirements  Generated  By  the  User.  All 
of  the  respondents  agreed  that  element  C5  was  not  necessary 
as  it  was  currently  written.  These  attitudes  were 
capsulized  in  Col  (Ret)  James  Lindenf elser ’ s  statement,  ‘The 
user  should  generate  the  need,  not  the  detailed  requirement* 
(Lindenf elser ,  1088).  Mr.  Joseph  Drelicharz  emphasized, 

‘The  user  doesn’t  necessarily  have  the  knowledge  and 
information  to  define  the  requirements  for  high  technology 
and  black  boxes*  (Drelicharz,  1988). 

Six  respondents  commented  that  element  C5  was  not 
necessary.  They  added  that  users  were  directed  by 
regulation  to  generate  the  Statement  of  Meed,  not  detailed 
requirements.  These  six  respondents  did  not  mention 
potential  modifications  that  could  make  this  element 
necessary  for  program  success. 

One  respondent.  Col  Ralph  Tourino,  believed  that 
element  C5  was  necessary  for  program  success.  He  stated 
that,  ‘User  involvement  in  requirements  definition  is  vital* 
(Tourino ,  1988) . 

The  two  remaining  respondents  believed  that  a  revised 
version  of  element  CS  was  necessary  for  program  success.  Lt 
Col  Paul  Huegel’s  comments  summarized  both  opinions. 

Colonel  Huegel  pointed  out  that  it  was  necessary  for  the 
requirements  process  to  be  driven  by  the  user.  If  it  was 
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not,  then  unnecessary  requirements  could  easily  creep  in. 

He  went  on  to  caution  that  requirements  generated  by  the 
user  in  terms  of  end  product  instead  of  need  could  lock  the 
procuring  agency  into  only  one  alternative.  Colonel  Huegel 
proposed  that  element  C5  be  revised  to  reflect  the  necessity 
of  having  users  generate  requirements  in  terms  of 
operational  need. 

It  was  found  that  element  C5  would  be  necessary  in 
revised  form.  This  conclusion  was  reached  because  the 
revision  suggested  by  Colonel  Huegel  was  in  line  with  the 
statements  of  the  six  respondents  who  determined  that  this 
element  was  not  necessary  as  it  was  written. 

Problem  Resolution  Impacts  Contractor  Payment 
Schedule .  The  respondent  opinions  concerning  the  necessity 
of  element  C6  were  equally  split.  Of  the  six  respondents 
who  commented  on  this  element,  three  believed  that  it  was 
not  necessary,  and  three  believed  that  it  was.  The 
following  statements  by  Mr.  Joseph  Drelicharz  summarized  the 
opinions  of  those  who  believed  that  element  C6  was  not 
necessary:  'Be  careful,  this  element  could  take  away  the 
contractor’s  motivation.  From  the  contractor’s  point  of 
view:  'All  of  these  safeguards!'  'Don’t  you  trust  me?' 
'Aren’t  we  a  team?''  (Drelicharz,  1968). 

The  following  statement  by  Col  Harry  Qillogly  III 
epitomized  the  opinions  of  the  respondents  who  believed  that 
element  C6  was  necessary,  'A  good  way  to  motivate  the 


96 


contractor*  (Qillogly,  1988).  Col  (Ret)  James  Lindenfelser 
provided  more  support  for  this  element  when  he  stated  that  a 
successful  program  had  to  have  a  cooperative  contractor,  and 
that  cooperation  could  be  achieved  through  this  element. 

After  further  evaluation,  a  similarity  between  the  two 
groups  was  Identified.  Both  groups  recognized  the  need  to 
generate  contractor  cooperation.  The  disagreement  centered 
around  achieving  that  cooperation.  If  the  specific  contract 
provision  was  replaced  with  the  general  need  met  by  that 
provision  then  both  groups  could  potentially  agree  that 
element  C6  was  necessary.  Therefore,  It  was  found  that 
element  C6  would  be  necessary  In  revised  form. 

Findings .  The  answers  to  Investigative  question 
bb  for  the  elements  in  the  Contract  category  are  summarized 
below: 

Cl.  Necessary. 

C2.  Necessary. 

C3.  Necessary. 

C4.  Revise  to:  Funds  budgeted  by  either  user  or  program 
office  to  support  participative  requirements. 

C5.  Revise  to:  Requirements  generated  by  the  user  In 
terms  of  operational  need. 

C6.  Revise  to:  Include  specific  contract  provisions  to 
encourage  contractor  cooperation. 

Teamwork .  The  group  consensus  was  that  teamwork  was 
necessary.  The  following  statements  epitomize  the  opinions 
of  most  of  the  respondents:  1.  *To  succeed,  the  program 
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manager  must  preach  and  practice  teamwork'  (Huegel .  1986), 

2.  'Integrating  all  functions  is  vital'  (Druyun,  1988),  and 

3.  'Teamwork  is  necessary*  (Christie,  1988;  Douglass,  1988). 
The  respondents  were  hesitant  to  address  the  necessity  of 
each  individual  element,  even  though  they  acknowledged  the 
necessity  of  the  category.  This  was  due  to  concerns  about 
potential  conflicts  of  interest,  historical  lack  of 
cooperation  between  the  government  and  the  contractor,  and 
specific  program  applicability.  This  statement  by  Colonel 
Gillogly  clearly  voiced  one  of  the  major  concerns  felt  by 
many  respondents,  'Teamwork  is  important,  but  remember  that 
some  displacement  is  needed  between  the  program  management 
team  and  the  contractor'  (Gillogly,  1988).  Another  area  of 
concern  was  revealed  when  Mr.  Daniel  Rak  stated,  'The  degree 
of  teamwork  necessary  and  appropriate  is  dependent  on  the 
type  of  contract'  (Rak,  1988).  The  experts  recognized  both 
the  benefits  and  the  obstacles  to  teamwork,  but  were 
hesitant  to  make  definitive  statements  about  the  specific 
elements . 

The  responses  to  the  second  interview  question, 
applicable  to  the  elements  listed  in  the  Teamwork  category, 
are  discussed  and  analyzed  in  the  following  seven 
subsections.  A  subsection  is  devoted  to  each  of  the  six 
elements  in  this  category.  The  final  subsection  will 
summarize  the  results.  Table  5.3  summarizes  the  findings 
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Table  5.3 


Category:  Teasmtork 


Summary  of  Responses  To  The  Interview  Question: 
Is  This  Element  Necessary  For  An 
Acquisition  Program  To  Be  Successful? 

(in  percentages,  eleven  respondents) 


Necessary 

No 

Not 

Af  ter 

Code 

Comment 

Necessary 

Revision 

Necessary 

X 

X 

X 

X 

conc«rnin^  the  necessity  of  the  elements  in  the  Teamwork 


category . 

All  Constituents  Involved  in  Planning  and 
Scheduling .  All  of  the  respondents  commenting  on  element  T1 
agreed  that  it  was  necessary  for  program  success.  The  group 
opinion  was  sununarized  by  Colonel  Tourino’s  statement  that 
involving  all  constituents  in  planning  and  scheduling  was 
very  good.  He  added  that  this  element  facilitated  the  need 
to  dig  in  and  get  a  meeting  of  the  minds.  Several 
respondents  also  voiced  comments  of  caution.  Mr.  Kemp 
cautioned  that  the  program  manager  must  be  sure  that  someone 
was  in  charge  of  the  process.  Mr.  Drelicharz  qualified  his 
approval  by  stating.  'This  element  is  necessary  as  long  as 
the  basic  baseline  is  already  nailed  down,  and  is  used  as 
the  basis  for  planning  and  scheduling*  (Drelicharz,  1988). 
Cognizant  of  the  cautions,  it  was  found  that  element  T1  was 
necessary  for  program  success. 

Lateral  Communication  Channels  Open.  Six 
respondents  addressed  this  element.  Of  these  six,  three 
believed  that  element  T2  was  necessary,  and  three  believed 
that  some  modification  was  required.  The  respondents  who 
desired  revision  did  not  argue  with  the  basic  element.  They 
all  agreed  that  lateral  communication  was  both  needed  and 
important.  The  concern  was  that  the  entire  issue  was  not 
addressed.  Mr.  Drelicharz  summarized  the  situation  when  he 
stated,  'This  statement  is  inadequate.  Communication  must 


100 


be  vertical,  lateral,  and  include  the  outside  chain  of 
command'  (Drelicharz,  1988). 

The  varying  opinions  between  the  two  groups  was 
evaluated.  The  respondents  who  identified  element  T2  as 
necessary  evaluated  it  in  terms  of  its  specific  contribution 
to  program  success.  They  did  not  evaluate  it  against  all  of 
the  other  communication  channels  that  a  successful  program 
should  have.  The  respondents  who  suggested  modification  of 
element  T2  agreed  that  it  was  necessary,  but  emphasized  that 
it  was  insufficient  as  stated. 

By  revising  this  element  both  groups  would  be 
satisfied.  It  was  foiind  that  element  T2  would  be  necessary 
in  revised  form.  It  was  also  recommended  that  the 
suggestion  to  keep  the  outside  chain  of  command  informed  be 
evaluated  for  addition  to  the  management  strategy. 

Frequent  Meetings  Used  to  Resolve  Differences. 

Five  of  the  eleven  respondents  addressed  this  element. 

Three  respondents  determined  that  element  T3  was  neceasary 
as  stated.  Colonel  Huegel  emphasized  the  importance  of  this 
element  when  he  stated,  'This  is  in  line  with  the  idea  of 
contractor/government  team  effort,  which  is  necessary' 
(Huegel ,  1988) . 

The  other  two  respondents  believed  that  element  T3  was 
insufficient.  The  following  statements  by  Colonel  Tourino 
sunusarize  the  opinions  of  the  respondents  requesting 
revision,  'Frequent  meetings  are  not  a  prescription  for 


success  themselves.  Meetings  are  important  to  keep  the 
channels  of  communication  open,  but  everyone  needs  to  know 
why  they  are  there,  and  the  action  items  need  to  be  tracked' 
(Tourino,  1988).  Lt  Col  Robert  Angeli  further  clarified  the 
required  modification  when  he  stated,  *It  is  especially 
important  to  have  an  agenda  for  each  meeting'  (Angeli, 

1988)  . 

All  of  the  respondents  addressing  element  T3  recognized 
the  need  for  frequent  meetings  to  resolve  differences. 
Several  respondents  also  noted  that  some  important  issues 
were  not  addressed.  It  was  foxind  that  element  T3  was 
necessary,  and  that  the  additional  items  noted  should  be 
evaluated  for  addition  to  the  management  strategy. 

Responsibilities  Accepted.  Hot  Assigned.  The 
majority  or  the  respondents  did  not  address  this  element. 

Of  the  four  respondents  that  did  comment,  one  believed  that 
element  T4  was  necessary  for  all  programs,  and  three 
believed  that  it  was  not  necessary  for  program  success. 
Colonel  Angeli,  who  believed  that  T3  was  necessary,  stated, 
'The  less  authoritative  method  of  trying  to  sell  the  task 
may  be  a  better  idea.  It  makes  a  lot  of  sense'  (Angeli, 
1988) .  The  other  three  respondents  all  agreed  that  it  was 
nice  to  assign  responsibilities,  but  not  necessary  for 


achieving  program  success.  The  group  opinion  was  summarized 
by  Mr.  Ira  Kemp’s  statement  that  the  decision  to  assign  or 
sell  responsibilities  depended  on  the  quality  of  the 


Individual,  the  practicality  of  staffing,  and  the  degree  the 
program  manager  trusts  his/her  people.  None  of  the  three 
respondents  stated  that  this  element  was  necessary  for 
program  success . 

This  element  was  found  to  be  not  necessary  for  program 
success.  Although,  it  was  recommended  that  program  managers 
allow  team  members  to  accept  responsibilities  instead  of 
assigning  them  wherever  practical. 

One  Face  to  Users  and  Management.  The  four 
respondents  that  addressed  element  T5  unanimously  agreed 
that  it  was  necessary  for  program  success.  Mr.  Drelicharz’s 
comment  that  this  element  was  vital  to  program  success 
epitomized  the  group  opinion.  This  element  was  found  to  be 
necessary . 

All  Functions  Involved  in  Problem  Resolution.  The 
majority  of  the  respondents  addressed  this  element.  Five 
identified  element  T6  as  necessary  for  program  success.  The 
necessity  of  this  element  was  summarized  by  Colonel  Huegel's 
statement,  'If  the  program  manager  does  not  work  with  the 
contractor  to  resolve  problems,  then  the  chances  of  poor 
program  execution  increase  accordingly  (Huegel ,  1986) . 

Two  respondents  commented  that  element  T6  was  suitable 
only  in  certain  situations,  and  not  necessary  for  program 
success.  The  following  comments  by  Mr.  Rak  summarized  both 
opinions  that  element  T6  was  not  really  necessary:  'This 
element  is  not  always  applicable.  It  depends  on  how  much 


103 


responsibility  the  government  wants  to  take  for  the  solution 
which  goes  back  to  what  the  contractor  is  being  paid  to  do* 
(Rak,  1988). 

The  majority  of  the  respondents  believed  that  this 
element  was  necessary.  Therefore,  it  was  found  that  element 
T6  was  necessary  for  program  success.  In  addition,  the 
program  manager  was  cautioned  to  continually  evaluate  the 
degree  of  responsibility  being  accepted  by  the  government  in 
problem  resolution. 

Findings .  The  answers  to  investigative  question 
5b  lor  the  elements  in  the  Teamwork  category  are  summarized 
below: 

Tl.  Necessary. 

T2.  Revise  to:  Both  lateral  and  vertical  communication 
channels  open. 

T3.  Necessary. 

T4 .  Not  Necessary. 

T5 .  Necessary. 

T6.  Necessary. 

Participative  Leadership.  The  group  consensus  was  that 
participative  leadership  was  necessary  for  program  success. 
The  opinions  of  the  nine  respondents  who  addressed  the 
elements  in  this  category  are  reflected  in  the  following 
statements:  1.  ’Program  managers  have  got  to  have  the  user 
on  their  team’  (Lindenf elser .  1988),  2.  'All  of  these  items 
are  absolutely  essential'  (Qillogly,  1988),  and  3.  'The  user 
must  be  intimately  involved.  By  involving  the  users,  they 
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understand  the  tradeoffs  that  have  been  made  and  the 
acceptance  rate  is  higher*  (Druyun,  1988). 

The  responses  to  the  second  interview  question 
regarding  the  elements  in  the  Participative  Leadership 
category  are  discussed  and  analyzed  in  the  following 
paragraphs.  The  results  will  be  synopsized  in  the  last 
paragraph.  Table  5.4  summarizes  the  findings  concerning  the 
necessity  of  the  elements  in  the  Participative  Leadership 
category . 

The  range  of  responses  to  elements  PI  and  P2  was 
limited.  Of  the  respondents  who  addressed  these  elements, 
one  believed  that  PI  and  P2  required  revision,  and  the  rest 
believed  that  PI  and  P2  were  necessary  as  stated.  Mr. 

Joseph  Orelicharz  believed  that  revision  was  necessary.  He 
justified  his  position  by  stating  that  users  generally 
didn’t  have  the  necessary  knowledge  base  to  be  involved  to 
the  extent  specified  by  elements  PI  and  P2.  The  other  eight 
respondents  addressing  elements  PI  and  P2  considered  them  to 
be  necessary  as  stated.  An  evaluation  of  the  comments 
applicable  to  these  two  elements  indicated  that  both 
elements  were  necessary  for  program  success. 

All  of  the  respondents  addressing  elements  P3 ,  P4 ,  P5 , 
and  P6  were  unanimous  in  their  approval.  It  was  found  that 
elements  P3 ,  P4 ,  P5 ,  and  P6  were  necessary  for  program 
success . 
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Table  5.4 


Category:  Participative  Leadership 


Summary  of  Besponses  To  The  Interview  Question: 
Is  This  Element  Necessary  For  An 
Acquisition  Program  To  Be  Successful? 

(in  percentages,  eleven  respondents) 


Code 

No 

CoBunent 

X 

Not 

Necessary 

X 

Necessary 

After 

Revision 

X 

Necessary 

X 

PI . 

18 

- 

9 

73 

P2. 

27 

- 

9 

64 

P3. 

27 

- 

- 

73 

P4. 

36 

- 

- 

64 

P5. 

36 

- 

- 

64 

P6. 

45 

- 

55 

106 


The  answers  to  investigative  question  5b  for  the 
elements  in  the  Partioipative  Leadership  category  are 


sumniarized  below: 

PI.  Necessary. 

P2 .  Necessary. 

P3.  Necessary. 

P4.  Necessary. 

P5.  Necessary. 

P6.  Necessary. 

Focus .  The  responses  received  to  the  second  interview 
question  for  the  elements  in  the  Focus  category  declared  the 
importance  and  necessity  of  these  elements.  The  following 
statements  by  Colonel  Tourino  expressed  the  sentiments  of 
every  respondent:  'These  are  right  on.  You  need  to  keep  the 
contractor  and  program  office  both  focused  on  the  heart  of 
the  problem*  (Tourino,  1988). 

The  following  four  subsections  discuss  and  analyze  the 
respondents'  comments  concerning  the  necessity  of  each  of 
the  three  elements  in  the  Focus  category.  A  subsection  is 
devoted  to  each  element.  The  results  will  be  summarized  in 
the  final  subsection.  Table  5.5  outlines  the  findings 
concerning  the  necessity  of  these  three  elements. 

No  Bells.  Whistles,  or  Additional  Fixes  Were 
Added .  Element  FI  was  unanimously  determined  to  be 
necessary  for  program  success.  Mr.  Christie  epitomized  the 
group’s  comments  when  he  stated  that  this  element  was 


Table  5.5 


Category:  Focus 


Suminary  of  Responses  To  The  Interview  Question: 
Is  This  Element  Necessary  For  An 
Acquisition  Program  To  Be  Successful'? 

(in  percentages,  eleven  respondents) 


Code 

No 

Comment 

X 

Not 

Necessary 

X 

Necessary 

Af  ter 
Revision 

X 

Necessary 

X 

FI. 

- 

- 

- 

100 

F2. 

9 

- 

9 

82 

F3. 

9 

91 
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•xcellent  and  absolutely  needed.  Mr.  Kemp  emphasized  the 
importance  of  evaluating  each  potential  addition  to  the 
program  to  ensure  that  it  was  necessary  for  the  program  to 
work,  versus  just  nice  to  have.  Due  to  the  unanimous 
approval,  it  was  found  that  element  FI  was  necessary  for 
program  success . 

Innovation  Restricted  to  the  Original  Program 
Scope .  Nine  of  the  ten  respondents  addressing  element  F2 
believed  that  it  was  necessary  for  program  success.  The 
importance  of  this  element  was  reflected  in  Colonel 
Qlllogly’s  statements,  'For  a  program  to  be  successful,  you 
can’t  let  the  baseline  be  destroyed.  This  is  a  very 
important  area*  (Gillogly,  1988).  Colonel  Qillogly  was  also 
the  respondent  who  believed  that  element  F2  required 
modification.  His  opinion  was  that  innovations  should  only 
be  dismissed  if  they  violated  the  baseline.  He  went  ovi  to 
explain  that  a  baseline  was  violated  only  if  the  schedule 
had  to  be  extended,  or  if  funds  were  not  available. 

Colonel  Qillogly’s  statements  were  discussed  with  an 
experienced  program  management  team  member.  It  was 
concluded  that  only  innovations  within  the  original  program 
scope  could  be  implemented  without  violating  the  baseline. 

In  addition,  an  overwhelming  number  of  respondents  commented 
that  element  F2  was  necessary.  They  did  not  indicate  that 
any  modifications  or  qualifications  were  needed.  Therefore, 
it  was  found  that  element  F2  was  necessary  for  program 
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success  as  currently  stated. 

Studies  Performed  by  Support  Contractors  Limited. 

The  respondents  addressing  element  F3  were  accordant  in 
their  approval.  Colonel  Tourino  summarized  the  opinions  of 
the  group  when  he  commented,  *It  is  important  to  be  tough  on 
additional  studies*  (Tourino,  1988).  Mr.  Drelicharz 
reiterated  the  necessity  for  this  element  when  he  stated 
that  program  managers  must  give  support  contractors 
sufficient  guidance,  and  tell  them  which  areas  to  focus  on. 

Minimal  analysis  was  required.  It  was  found  that  element  F3 
was  necessary  for  program  success. 

Findin<<s .  The  answers  to  investigative  question 
Sb  for  the  elements  in  the  Focus  category  are  summarized 
below: 

! 

FI.  Necessary. 

F2 .  Necessary. 

F3 .  Necessary. 

I 

Contractor  and  System  Monitoring.  The  range  of 
responses  to  this  category  and  its  composing  elements  was 
broad.  So^en  respondents  commented  on  the  current  efforts 

I 

to  streamline  the  DOD  acquisition  process  when  they  saw  this 
category.  The  DOD  streamlining  goals  are  outlined  below  to 
help  the  reader  understand  the  impact  these  initial  comments 

I 

had  on  the  subsequent  interpretation  of  the  elements  in  this 
category.  The  DOD  streamlining  goals  according  to  Lt  Col 

I 
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Robert  R.  Angeli,  Course  Director  for  Policy,  Defense 
Systems  Management  College,  follow: 

1.  Do  not  over-specify  requirements.  Specify  only 
those  requirements  that  meet  an  actual  need. 

2.  Shorten  the  chain  of  command  from  the  program 
manager  to  the  Secretary  of  Defense  and  to  the  Service 
Acquisition  Executive. 

3.  Do  not  tell  the  contractor  how  they  should 
accomplish  the  specifications.  Let  the  contractor  make 
his/her  own  decisions  on  how  best  to  do  it. 

4.  Get  the  'boilerplate*  clauses  out  of  there.  Start 
at  ground  zero  and  add  only  the  clauses  that  are 
appropriate . 

Of  the  seven  respondents  who  commented  on  DOD 
acquisition  streamlining,  five  did  not  address  the 
individual  elements  listed  under  the  Contractor  and  System 
Monitoring  category.  Their  comments  are  epitomized  in  the 
following  statements:  ‘The  best  thing  to  do  is  to  give  the 
contractor  the  requirement,  and  then  back  off.  The 
government  needs  to  take  the  regulations  off  of  both  the 
program  manager’s  back  and  the  contractor’s  back"  (Christie, 
1988).  The  consensus  of  these  five  respondents  appeared  to 
be  that  the  elements  listed  under  the  Contractor  and  System 
Monitoring  category  were  ‘requirements  on  the  backs  of 
program  managers  and  contractors.' 

Evaluation  of  the  elements  in  the  Contractor  and  System 
Monitoring  category  revealed  that  these  elements  were  not 
within  the  domain  of  practices  that  DOD  was  attempting  to 
eliminate.  Element  SI  addressed  the  need  to  ensure  that  the 


product  performed  as  specified  in  the  contract.  Elements  S2 


and  S3  addressed  the  measures  of  product  performance 
necessary  to  facilitate  subsequent  product  evaluation. 
Element  S4  and  S5  addressed  the  need  for  the  program  manager 
to  monitor  the  degree  of  schedule  and  performance  risk.  The 
conclusion:  none  of  the  elements  in  the  Contractor  and 
System  Monitoring  category  were  candidates  for  DOD 
streamlining. 

To  reiterate,  five  of  the  seven  respondents  who 
commented  on  OOD  acquisition  streamlining  did  not 
specifically  evaluate  each  element.  In  addition,  they 
appeared  to  express  disapproval  of  the  category  as  a  whole. 
The  other  two  respondents  went  on  to  evaluate  each 
individual  element.  Their  attitudes  concerning  these 
elements  after  evaluation  was  summarized  in  Colonel  Angeli’s 
statement,  'The  current  push  in  DOD  for  streamlining  does 
not  really  affect  any  of  these  items,  nor  negate  the  need 
for  them'  (Angeli,  1988).  Both  of  these  respondents 
determined  that  all  of  the  elements  in  this  category  were 
necessary  for  program  success.  The  confusion  and  subsequent 
negative  attitude  seemed  to  be  caused  by  the  category  title. 

Further  investigation  revealed  that  DOD  acquisition 
personnel  interpreted  the  phrase  'contractor  monitoring*  to 
mean  the  regulatory  requirements  that  the  streamlining  goals 
were  attempting  to  reduce  or  to  apply  on  a  program  by 
program  basis.  It  was  concluded  the  category  title  required 
revision.  A  new  category  title.  Progress  and  Performance. 
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««ft8  identified  that  better  described  the  purpose  of  elements 
SI  through  S5 ,  and  eliminated  the  potentially  misleading 
phrase.  It  should  be  noted  that  the  other  respondents, 
excepting  the  five  respondents  discussed  in  preceding 
paragraphs,  addressed  each  element  individually. 

The  following  subsections  discuss  and  analyze  the 
responses  received  to  the  second  interview  question.  A 
subsection  is  devoted  to  each  of  the  five  elements  in  the 
Contractor  and  System  Monitoring  category.  The  final 
subsection  will  synopsize  the  results.  Table  5.6  summarizes 
the  findings  concerning  element  necessity  that  are 
applicable  to  this  category.  To  facilitate  this 
summarization  the  Uncertain.  Possibly  Necessary  column  was 
added . 

Actual  Demonstration  of  Requirement  Before  Sign- 
off.  The  consensus  of  the  respondents  addressing  element  SI 
was  that  it  was  both  important  and  necessary.  Colonel 
Qillogly  suBunarized  the  group  beliefs  when  he  stated,  ‘Very 
important.  Definitely  worth  your  time.  It  is  not 
necessarily  micromanagement  to  hold  the  contractor 
responsible'  (Qillogly,  1988).  He  went  on  to  add  that  when 
he  was  a  program  manager  he  ensured  that  every  requirement 
was  validated.  Mr.  Drelicharz  agreed  that  element  SI  was 
necessary,  but  he  also  felt  compelled  to  caution  that 
program  managers  must  be  sure  and  look  at  the  whole  picture 
when  determining  if  a  requirement  has  been  met  or  not. 
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Tabla  5.6 


Category:  Contractor  and  System  Monitoring 


Suamary  of  Responses  To  The  Interview  Question: 
Is  This  Element  Necessary  For  An 
Acquisition  Program  To  Be  Successful? 

(in  percentages,  eleven  respondents) 


Code 

No 

Comment 

X 

Not 

Necessary 

X 

Uncertain , 

Possibly 

Necessary 

X 

Necessary 
Af  ter 
Revision 

X 

Necessary 

X 

SI. 

55 

- 

- 

- 

45 

S2. 

45 

- 

- 

- 

55 

S3. 

55 

- 

- 

- 

45 

S4. 

36 

- 

18 

- 

45 

S5. 

45 

9 

45 
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on  unanlotous  approval,  and  cognizant  of  the  cautions, 
it  was  found  that  element  SI  was  necessary  for  program 


success . 

Technical  Performance  Measures  Established  Up 
Front .  All  of  the  respondents  addressing  element  S2  stated 
that  it  was  necessary  for  program  success.  The  following 
statement  by  Colonel  Tourino  epitomized  the  group  comments, 
'If  technical  performance  measures  are  not  in  place  at  the 
beginning  then  the  cows  are  already  out  of  the  barn  by  the 
time  you  get  the  data*  (Tourino,  1988).  Minimal  analysis 
was  required.  Element  S2  was  found  to  be  necessary  for 
program  success. 

RMA  Established  Up  Front,  and  Achievable.  The 
respondents  addressing  element  S3  unanimously  believed  that 
establishing  repairabi 1 i ty ,  maintainability,  and 
availability  (RMA)  standards  in  accordance  with  this  element 
made  sense.  The  following  comments  emphasized  the  necessity 
of  element  S3:  'It  is  important  for  the  program  manager  to 
ensure  that  RMAs  are  achievable  before  they  are  put  down  in 
the  contract.  If  they  are  not,  then  the  program  runs  into 
trouble'  (Angeli,  1988).  Based  on  the  respondent  comments, 
element  S3  was  found  to  be  necessary  for  program  success. 

All  Problems  Categorized  and  Visible  in  INFO 
Database .  The  comments  concerning  element  S4  were  all 
positive.  Mr.  Drelicharz  summarized  the  group  opinion  when 
he  stated  that  element  S4  was  a  good  idea  to  solve  an  age- 


old  problam.  Two  of  the  reepondenta  were  enthusiastic  about 
the  element,  but  were  unable  to  determine  if  it  was 


necessary  for  program  success.  The  comments  concerning  this 
element  were  evaluated.  Five  of  the  seven  respondents  who 
addressed  this  element  determined  that  it  was  necessary. 

The  two  respondents  who  were  hesitant  to  make  a 
determination  made  no  negative  comments.  Therefore,  element 
S4  was  found  to  be  necessary  for  program  success. 

Mumeroua  Indicators  of  Program  Progress  Used.  The 
range  of  responses  to  element  S5  was  limited.  Five 
respondents  believed  that  this  element  was  necessary  for 
program  success.  Their  opinions  were  epitomized  by  Colonel 
Angeli's  comment  that  this  element  was  good  because  it  would 
help  to  highlight  contradictory  information  and  potential 
problems.  Lt  Col  Paul  Huegel  believed  that  element  S5  was 
Interesting  and  maybe  even  necessary.  He  could  not  state 
with  certainty  that  this  element  was  necessary  for  program 
success.  After  evaluating  the  responses,  it  was  found  that 
element  S5  was  necessary  for  program  success. 

Findings .  The  answers  to  investigative  question 
5b  for  the  elements  in  the  Contractor  and  System  Monitoring 
category  are  Summarized  below: 

51.  Necessary. 

I 

52.  Necessary. 

53.  Necessary. 

54 .  Necessary. 
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S5.  Nacessary. 

Taating  and  Tpainin<t.  The  group  consensus  was  that 
both  testing  and  training  were  essential  for  program 
success.  The  opinions  of  the  ten  respondents  who  addressed 
the  elements  in  this  category  are  reflected  in  the  following 
statement  by  Ms.  Darleen  Druyun.  'A  disciplined  test  process 
defined  up  front  will  increase  the  chances  of  program 
success*  (Druyun,  1988). 

The  responses  to  the  second  interview  question 
regarding  the  elements  in  the  Testing  and  Training  category 
are  discussed  and  analyzed  in  the  following  paragraphs.  The 
results  will  be  synopsized  in  the  last  paragraph.  Table  5.7 
sununarizes  the  findings  concerning  the  necessity  of  the 
elements  in  the  Testing  and  Training  category. 

There  was  only  one  respondent  who  did  not  comment  on 
elements  A1 ,  A2 ,  and  A3.  The  respondents  who  did  address 
these  elements  unanimously  believed  that  elements  A1 ,  A2 , 
and  A3  were  essential  for  program  success.  The  following 
statements  capsulize  the  group’s  sentiments:  'You  have  got 
to  know  whether  the  criteria  are  met  as  soon  as  possible* 
(Kemp,  1988),  *It  is  important  to  find  problems  early, 
before  the  program  goes  over  budget  and  while  there  is  still 
time  to  fix  them*  (Christie,  1988),  'Without  unstructured 
testing  a  program  goes  into  production  with  faulty  software* 
(Christie,  1988),  and  'User  involvement  is  absolutely 
imperative  in  this  area*  (Lindenf elser ,  1988).  The 
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Table  5.7 


Category:  Testing  and  Training 


Summary  of  Responses  To  The  Interview  Question: 
Is  This  Element  Necessary  For  An 
Acquisition  Program  To  Be  Successful? 

(in  percentages,  eleven  respondents) 


Code 

No 

Comment 

X 

Not 

Necessary 

X 

Necessary 

Af  ter 
Revision 

X 

Necessary 

X 

A1 . 

9 

- 

- 

100 

A2. 

9 

- 

- 

91 

A3. 

9 

- 

- 

91 

A4. 

9 

- 

9 

82 

A5. 

18 

82 
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evaluation  of  responses  clearly  indicated  that  elements  AI , 
A2 ,  and  A3  were  necessary  for  program  success. 

Eight  of  the  nine  respondents  addressing  element  A4 
believed  that  it  was  vital  for  program  success.  The 
following  statement  by  Mr.  Ira  Kemp  epitomized  their 
comments,  ‘The  most  crucial  part  of  any  system!  Without 
trained  operators  and  maintenance  personnel  a  system  is 
useless*  (Kemp.  1988).  Mr.  Joseph  Drelicharz  agreed  that 
element  A4  was  necessary,  but  he  also  believed  that  it  was 
insufficient.  He  stated  that  this  element  was  necessary, 
but  that  it  should  include  much  more  than  having  personnel 
trained.  He  further  explained  that  the  site  also  had  to  be 
prepared  and  ready  to  support  the  system  before  it  was 
delivered. 

The  evaluation  included  both  the  comments  received 
concerning  element  A4 ,  and  the  data  collected  during  the 
case  study  of  the  Host  computer  system.  Based  on  the  Host 
program  management  team  comments,  and  on  the  observed  detail 
going  into  Host  site  preparations  it  was  concluded  that  Mr. 
Drelicharz's  suggested  modification  should  be  evaluated  for 
addition  to  the  management  strategy.  It  was  also  found  that 
element  A4  was  necessary  for  program  success  as  stated. 

All  of  the  respondents  addressing  element  AS  agreed 
that  it  was  necessary  for  program  success.  The  following 
statement  by  Mr.  Drelicharz  summarized  the  group’s  comments, 
‘Yes,  this  is  needed  continuously  and  forever'  (Drelicharz, 
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1988).  Minimal  analysis  was  pequired.  Elamant  AS  was  found 
to  ba  nacsssary  for  program  succass. 

Tha  answars  to  investigative  question  5b  for  the 
alamants  in  tha  Testing  and  Training  category  are  summarized 
below: 

A1 .  Necessary. 

A2.  Necessary. 

A3.  Necessary. 

A4 .  Necessary. 

AS.  Necessary. 

Element  Applicability  to  the  POD  Acquisition  Process 

This  section  presents  the  range  and  general  sentiment 
of  the  responses  received  to  the  second  interview  question. 
This  information  was  used  to  answer  investigative  questions 
Sc,  5d ,  5e,  and  5f.  The  third  interview  question  concerned 
only  those  management  strategy  elements  which  the  respondent 
had  identified  as  necessary  for  program  success  as  stated. 
For  each  of  the  elements  categorized  as  necessary,  the 
respondent  was  required  to  determine  if  that  element  was 
currently  used,  potentially  applicable,  or  not  applicable  to 
the  DOO  acquisition  process.  The  purpose  of  the 
investigative  questions  was  to  methodically  evaluate  the 
elements  in  the  hypothesized  management  strategy,  and  to 
eliminate  those  that  were  not  necessary  for  program  success 
and/or  not  applicable  to  DOD. 
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In  answering  the  second  interview  question  addressing 
element  necessity,  several  elements  were  found  to  be 
necessary  after  modification,  and  one  element  was  found  to 
be  not  necessary  for  program  success.  Table  5.8  summarizes 
the  findings  to  investigative  question  5b  which  provide  the 
basis  for  further  investigation.  The  one  element  found  to 
be  not  necessary  did  not  need  to  be  addressed  in  this 
section  because  it  had  already  been  eliminated  from  the 
proposed  management  strategy.  In  addition,  the  elements 
found  to  require  revision  did  not  need  to  be  addressed  in 
this  section  because  the  respondents  believed  that  the 
modified  element  tvould  be  necessary  for  program  success  and 
applicable  to  the  DOD  acquisition  process.  Without 
modification,  the  recommending  respondents  believed  that  the 
element  vrould  not  be  necessary  for  program  success,  which 
would  eliminate  it  from  the  proposed  management  strategy. 
This  section  addresses  only  those  elements  listed  in  Table 
5.8  that  were  found  to  be  necessary  for  program  success. 

The  respondents  primarily  focused  on  addressing  element 
necessity  and  provided  general  responses  to  the  third 
interview  question.  They  specifically  addressed  only  those 
elements  which  they  believed  had  limited  applicability.  The 
majority  of  the  respondents  answered  the  third  interview 
question  with  a  general  comment  at  the  conclusion  of  the 
interview.  This  section  is  further  divided  into  seven 
sections.  A  section  is  devoted  to  each  of  the  six 
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Tablft  5.8 


Siiamary  of  Findings  to  Investigative  Question  5b: 
Identifying  Elements  Necessary  for  Program  Success 


Element 

Finding 

Contract : 

Cl. 

Necessary 

C2. 

Necessary 

C3. 

Necessary 

C4. 

Revision  Required 

C5. 

Revision  Required 

C6. 

Revision  Required 

Teamwork : 

T1 . 

Necessary 

T2. 

Revision  Required 

T3. 

Necessary 

T4. 

Not  Necessary 

T5. 

Necessary 

T6. 

Necessary 

Participative 

Leadership: 

PI. 

Necessary 

P2. 

Necessary 

P3. 

Necessary 

P4. 

Necessary 

P5. 

Necessary 

P6. 

Necessary 

Focus : 

FI . 

Necessary 

F2. 

Necessary 

F3. 

Necessary 

Contractor  and  System  Monitoring: 


SI . 

Necessary 

S2. 

Necessary 

S3. 

Necessary 

S4. 

Necessary 

S5. 

Necessary 

ting  and  Training 

Al. 

Necessary 

A2. 

Necessary 

A3. 

Necessary 

A4. 

Necessary 

A5. 

Necessary 
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c«t«gori«8  in  the  management  strategy.  The  final  section 
summarizes  the  concluding  comments  of  the  respondents. 

Contract .  The  respondents  who  determined  that  the 
elements  Cl.  C2 ,  and  C3  were  necessary  for  program  success 
were  unanimous  in  their  findings.  They  determined  that 
these  elements  were  currently  used  in  DOD ,  and  were 
applicable  to  all  DOD  programs.  The  following  statements  by 
Colonel  Huegel  summarized  their  comments: 

It  is  most  important  to  define  user  requirements 
through  the  Statement  of  Need.  It  is  next  most 
important  to  set  up  the  contract  to  facilitate  program 
success.  Air  Force  Systems  Command  is  very  proud  of 
their  efforts  in  this  area.  The  Acquisition  Strategy 
Panel  and  Acquisition  Strategy  Principles  facilitate  up 
front  planning  for  contract  type  and  program 
management.  (Huegel.  1988) 

Several  of  the  respondents  specifically  addressed 
elements  Cl.  C2 .  and  C3.  Their  comments  added  support  to 
the  findings  that  these  elements  were  currently  used  in  DOD 
and  were  globally  applicable.  Concerning  element  Cl,  Ms. 
Darleen  Druyun  stated  that  Headquarters  AFSC  was  preparing 
an  Award  Fee  Guide  to  increase  the  combined  use  of  award 
fees  and  incentive  fees  in  the  Air  Force.  When  addressing 
element  C2  Mr.  Ira  Kemp  commented.  'Of  course  this  is 
applicable.  It  is  always  effective  to  build  on  the  learning 
curve  and  have  the  core  team  move  from  site  to  site'  (Kemp, 
1988).  Colonel  Tourino  made  the  following  statements 
concerning  element  C3 ,  'There  has  been  a  revolution  at  AFSC 
in  the  last  year.  There  is  a  nuijor  push  for  realistic 
schedules'  (Tourino,  1988).  In  addition,  the  need  to 


establish  realistic  program  leadtimes  was  also  identified  in 
AFSC  Regulation  550-21. 

Teamwork .  The  range  of  responses  to  the  third 
interview  question  for  the  elements  determined  to  be 
necessary  for  program  success  was  very  narrow.  The 
consensus  of  the  respondents  was  that  elements  Tl,  T3 ,  T5 , 
and  T6  were  currently  used  in  DOD ,  and  were  applicable  to 
all  DOD  programs.  The  respondents  were  also  very  concerned 
about  the  current  condition  of  the  relationship  between  the 
government  and  contractor.  The  following  statements 
summarize  both  the  group’s  beliefs  that  Teamwork  was 
applicable  to  DOD.  and  their  concerns: 

1.  Teamwork  makes  sense,  we  hope  in  DOD  that  these 
good  leadership  techniques  are  applied.  (Angeli,  1988) 

2.  In  DOD  the  teamwork  between  the  government  and  the 
contractor  is  directly  affected  by  external  factors. 
Every  time  DOD  gets  bad  publicity,  the  relationship 
between  the  service  and  the  contractor  gets  strained. 
(Lindenf elser ,  1988) 

3.  Every  day  the  Air  Force  and  the  contractor  get 
further  and  further  away  from  being  a  good  team. 
Industry  does  not  want  to  do  business  with  us  because 
we  are  perceived  as  a  miserable  customer.  Industry  is 
also  concerned  with  how  the  public  perceives  them. 

This  area  is  of  critical  concern.  The  trend  is  in  the 
wrong  direction.  The  pentagon  is  trying  to  turn  this 
around.  (Douglass,  1988) 

The  need  for  teamwork  was  also  noted  in  General  Bernard  P. 
Randolph’s  Coimnander ’ s  Policies.  In  AFSC  Regulation  550-2, 
Command  Goals,  he  stated,  'To  be  successful,  it  is  essential 
that  we  think  and  act  as  a  team'  (Department  of  the  Air 
Force,  I987e) .  He  went  on  to  add  that  acquisition 
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excellence  was  maintained  by  fostering  productive  teamwork. 
In  AFSC  Regulation  550-4,  Teamwork .  General  Randolph 
emphasized  that  teamwork  across  the  functional  boundaries  of 
the  procuring  government  organization  was  viewed  as  very 
important . 

The  comments  of  the  respondents  who  specifically 
addressed  elements  Tl,  T3 ,  and  T5  are  provided  below.  These 
comments  provided  additional  support  to  the  findings  that 
the  elements  of  concern  were  currently  used  in  DOD  and  were 
globally  applicable.  Colonel  Tourino  made  the  following 
statement  regarding  element  Tl,  ’Some  program  offices  don’t 
have  time  to  do  this,  but  this  is  the  better  way'  (Tourino, 
1988).  Mr.  Kemp  added,  ’Probably  a  good  lesson  learned, 
applicable  to  the  Air  Force,  but  something  that  DOD  usually 
does  not  do  well’  (Kemp,  1988).  Concerning  element  T3 , 
Colonel  Huegel  summarized  the  group  beliefs  when  he  stated, 
’The  idea  of  contractor/government  team  effort  has  always 
been  promoted  at  AFSC’  (Huegel,  1988).  Mr.  Drelicharz 
commented  that  element  T5  was  a  DOD  goal,  but  qualified  his 
response  with  the  following  statement,  ’In  DOD  this  is  a 
tough  thing  to  do.  There  are  too  many  special  interest 
groups  trying  to  give  the  contractor  guidance.  The  program 
manager  must  be  continually  reminding  the  contractor  that 
these  outside  groups  can't  change  the  agreement  already 
established  with  the  primary  contracting  officer' 


(Dr«licharz,  1988).  There  were  no  specific  examples  of 
element  T6  application  to  OOD  programs. 

Participative  Leadership.  All  of  the  elements  in  this 
category  were  determined  to  be  necessary  for  program 
success.  The  respondents  were  also  unanimous  in  their 
beliefs  that  these  elements  were  all  currently  used  in  DOD 
and  were  applicable  to  all  programs.  The  respondents  made 
few  comments  concerning  each  specific  element.  In  answering 
the  third  interview  question  for  each  element  the 
respondents  used  the  phrase  'definitely  applies*  frequently. 
In  addition,  their  comments  concerning  the  Participative 
Leadership  category  made  their  beliefs  clear.  Mr.  Rak 
pointed  out  that  DOD  involved  users  and  that  most  Air  Force 
programs  assigned  a  user  representative  full  time  to  the 
System  Program  Office.  General  Randolph  also  made  his 
support  of  participative  leadership  clear.  In  AFSC 
Regulation  550-24,  Commander’s  Policies.  Acquisition  Team 
Concept .  he  stated: 

My  experience  in  the  acquisition  business  has 
shown  me  conclusively  that  System  Program  Offices  that 
have  close,  informal  working  relationships  with  their 
using  command  (customers)  consistently  achieve  the  best 
results  in  the  Air  Force.  (Department  of  the  Air 
Force ,  1987a) . 

General  Randolph  further  addressed  the  issue  in  AFSC 
Regulation  550-21,  Commander’s  Policies.  Acquisition 
Strategy .  where  he  provided  the  following  additional 
direction  to  program  managers: 
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Se«k  expert  advice  from  wherever  necessary  to 
formulate  sound  strategy.  ...I  expect  the  user,  the 
supporter,  Headquarters  Air  Force,  and  the  Secretariat 
to  be  invited  to  participate.  I  expect  the  program 
directors  to  develop  the  best  strategy  possible  and, 
before  strategy  decisions  are  set  in  concrete,  to 
listen  to  the  advice  of  the  best  talent  available  in  an 
open-minded,  communicative  manner.  (Department  of  the 
Air  Force,  1987b} 

Focus .  The  consensus  among  the  respondents  was  that 
the  three  elements  in  the  Focus  category  were  vital  for 
program  success.  The  respondents  addressing  these  elements 
were  also  unanimous  in  their  determinations  that  FI.  F2 ,  and 
F3  were  currently  used  in  DOD .  and  were  applicable  to  all 
programs.  When  evaluating  element  F3 .  one  respondent 
commented  that  he  had  never  seen  the  concept  written  down 
before,  although  it  was  currently  applied.  General  Randolph 
also  addressed  this  area  in  his  Commander's  Policies.  In 
AFSC  Regulation  550-22,  Program  Baseline  Management,  he 
stated,  *0ur  ability  to  establish  realistic  baselines  and  to 
stick  to  them  is  a  key  element  of  acquisition  excellence* 
(Department  of  the  Air  Force,  1987d) . 

The  following  comments  addressing  the  specific  elements 
provide  additional  support  to  the  findings  that  these 
elements  were  currently  used  in  DOD  and  applicable  to  all 
programs.  Colonel  Huegel  made  the  following  statements 
while  evaluating  element  FI: 

Program  stability  is  vital.  Headquarters  AFSC  has 
taken  action  to  stabilize  r''.quirements .  Currently 
Headquarters  is  pushing  to  get  a  single  integrated 
program  baseline  that  the  program  manager,  program 
executive  officer,  and  service  acquisition  executive 
(SAE)  sign  up  to.  This  will  establish  commitment  and 
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acaountability  becausa  no  baseline  changes  will  be 
allowed  unless  approved  the  SAE.  (Huegel ,  1968) 

The  response  to  element  F2  is  epitomized  by  the 

comments  of  Mr.  Christie  and  Mr.  Drelicharz  who  stated  that 

the  concept  was  laudable,  but  hard  to  do  because  so  ntany 

external  organizations  could  interfere  with  that  program. 

Concerning  element  F3 ,  Mr.  Drelicharz  sununarized  the  group’s 

beliefs  regarding  this  element  when  he  stated,  'In  DOD  it  is 

important  to  give  the  support  contractors  sufficient 

guidance  and  to  tell  them  which  areas  to  focus  on* 

(Drelicharz,  1988). 

Contractor  and  System  Monitoring.  The  respondents 
addressing  elements  S2 ,  S3,  and  S4  were  unanimous  in  their 
responses  to  the  third  interview  question.  These  three 
elements  were  found  to  be  currently  used  in  DOD,  and 
applicable  to  all  programs.  The  following  statement  by  Mr. 
Rak  supported  these  findings,  'The  degree  [of  contractor  and 
system  monitoring]  required  depends  on  the  program,  but  it 
is  essential  to  every  program  irregardless  of  degree'  (Rak, 
1988).  The  majority  of  the  respondents  specifically 
addressed  each  element  in  response  to  the  third  interview 
question.  Representative  comments  are  presented  below. 

The  respondents  addressing  element  S2  found  it  to  be 
currently  used,  but  also  noted  that  it  was  hard  to 
accomplish.  Colonel  Huegel  pointed  out  that  a  new 
requirements  process  called  TEMP  had  been  instituted  by 
Headquarters  AFSC  to  assist  program  managers  establish 
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testing  specifics,  which  include  technical  performance 
measures,  up  front.  Colonel  Huegel  also  noted  that  TEMP 
would  help  with  accomplishing  element  S3.  Colonel  Angel i 
commented  that  element  S3  was  applicable  to  DOD,  but  added, 
'Perhaps  DOD  has  to  focus  on  this  area  a  bit  more*  (Angeli, 
1988).  The  following  statements  by  Colonel  Gillogly 
epitomizes  the  group’s  comments,  ‘The  program  manager  should 
only  promise  what  good  analytical  checks  of  the  state-of- 
the-art  technology  indicate  are  achievable.  This  is 
especially  crucial  when  procuring  aircraft,  because  the 
environment  is  unknown  and  many  Judgement  calls  must  be 
made*  (Qillogly,  1988). 

When  evaluating  element  S4  the  respondents  commented 
that  it  was  a  good  management  technique,  but  they  also  noted 
that  it  was  hard  to  implement  in  DOD.  These  attitudes  are 
summarized  by  Mr.  Christie’s  comment  that  the  system  program 
office  and  the  user  were  afraid  that  the  program  might  be 
cut  if  the  problems  were  visible.  Colonel  Gillogly  added 
support  to  the  finding  that  element  S4  was  currently  used  in 
DOD  when  he  commented  that  the  Air  Force  uses  a  system 
similar  to  INFO. 

The  respondents  addressing  elements  SI  and  S5  were 
dissenting  in  their  answers  to  the  third  interview  question. 
One  group  believed  that  these  elements  were  applicable  to 
all  programs,  and  the  other  group  believed  that  these 
elements  were  applicable  to  software  intensive  programs 
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only.  Colonel  Angeli'a  comments  summarized  the  beliefs  that 
SI  was  globally  applicable.  He  stated.  *DOD  is  currently 
doing  this.  In  the  Demonstration/Validation  phase  of  system 
acquisition  there  is  a  new  requirement  to  have  a  competitive 
prototype  flyoff*  (Angeli,  1988).  The  following  comments  by 
Colonel  Gillogly  countered  those  made  by  Colonel  Angeli,  'In 
a  software  intensive  program  demonstration  is  possible.  In 
other  procurements  it  is  not.  Prototyping  is  nice  because 
it  reduces  risk,  but  sometimes  ‘fly  before  you  buy'  doesn’t 
work'  (Qillogly,  1988).  Colonel  Qillogly  then  stated  that 
the  Air  Force  currently  had  a  mechanism  to  accomplish 
element  SI  called  the  Critical  Design  Review,  which  refers 
to  what  Colonel  Angeli  addressed  above.  Based  on  this  final 
comment  it  was  found  that  element  SI  was  currently  used  by 
DOD,  and  it  was  applicable  to  all  programs.  The  difference 
between  programs  would  be  in  bow  the  requirement  was 
demonstrated,  not  in  the  need  to  demonstrate  requirements. 

Colonel  Gillogly  believed  that  element  S5  was  not 
necessary  for  all  programs.  He  stated  that  the  necessity  of 
this  element  depended  on  the  type  of  system  being  procured 
and  on  the  type  of  contract.  He  added  that  close  monitoring 
was  vital  for  software  intensive  programs  like  Host.  On  the 
other  side  of  the  issue,  the  other  respondents  believed  that 
element  S5  was  applicable  to  all  programs.  The  following 
statements  by  Mr.  Drelicharz  summarized  this  group’s 
comments:  'I  want  to  see  this  in  every  program  office  and  in 
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th«  Hsadquartera  staff  office.  This  is  important  and 
necessary...'  (Drelicharz,  1968).  The  comments  by  Colonel 
Qillogly  concerning  the  limited  applicability  of  this 
element  were  evaluated  in  conjunction  with  the  comments  from 
the  other  respondents.  It  was  concluded  that  this  element 
was  applicable  to  all  programs.  The  following  statements  by 
Colonel  Tourino  supported  this  conclusion,  'It  is  important 
for  program  managers  to  get  their  own  data  and  to  do  their 
own  analysis  in  parallel  [with  the  contractor’s  analysis]' 
(Tourino,  1968).  Therefore  it  was  found  that  element  S5  was 
currently  used  by  DOD ,  and  was  applicable  to  all  DOD 
prograaas . 

Testing  and  Training.  The  respondents  who  determined 
that  the  elements  in  the  Testing  and  Training  category  were 
necessary  for  program  success  were  unanimous  in  their 
findings.  They  determined  that  these  elements  were 
currently  ussd  in  DOD,  and  that  they  were  applicable  to  all 
DOD  programs.  The  following  statements  summarized  their 
cootments : 

1.  According  to  AFSC  Acquisition  Strategy  Principles, 
a  disciplined  test  process  must  be  defined  up  front. 

If  it  is,  the  result  will  be  executable  test 
objectives,  measurable  test  criteria,  and  increased 
chance  of  program  success.  (Druyun,  1988;  Office  of 
the  Chief  of  Staff  for  Contracts,  1988:3) 

2.  Philosophically  these  elements  are  in  league  with 
DOD  objectives.  In  actuality,  DOD  is  having  a  hard 
time  meeting  these  elements.  Testing  especially  has 
been  under  scrutiny.  DOD  has  a  testing  agency  with  an 
independent  tester  in  each  service.  (Lindenf elser , 

1988) 


131 


Sever*!  of  the  respondents  specifically  addressed  these 
eleoients.  Their  comments  added  support  to  the  findings  that 
elesients  A1  ,  A2 ,  A3,  A4 ,  and  AS  were  currently  used  in  DOD , 
and  that  they  were  applicable  to  all  programs.  The 
responses  concerning  element  A1  are  summarized  in  the 
following  statements  by  Colonel  Angeli,  ‘Although  DOD  should 
do  this,  and  is  supposed  to  do  this,  it  is  frequently  not 
done.  Both  program  stanagers  and  users  are  afraid  of  finding 
problems,  and  tend  to  downplay  problems.  Colonel  Huegel 
again  commented  that  TEMP,  a  new  requirements  process,  was 
being  instituted  at  Headquarters  AFSC .  He  also  stressed, 
'Headquarters  AFSC  is  currently  putting  a  lot  of  emphasis  on 
getting  an  early  start  on  testing*  (Huegel,  1986).  The 
findings  concerning  element  A1  were  supported  by  the 
following  directives  from  General  Randolph  in  his 
CoBuaander * s  Policies.  ‘Program  directors  will  maintain  early 
and  continual  involvement  with  the  Air  Force  Operational 
Teat  and  Evaluation  Center...'  (Department  of  the  Air  Force, 
1987c)  . 

The  respondents  who  determined  that  element  A2  was 
necessary  unanimously  believed  that  it  was  currently  used  by 
DOD,  and  that  it  was  globally  applicable.  In  addition,  many 
of  them  were  compelled  to  add  some  words  of  caution.  The 
following  statements  summarized  the  respondent’s  comments 
and  concerns : 

1.  The  Air  Force  tries  to  approach  unstructured 

testing  in  a  more  logical  method.  You  can  'crash'  a 
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computer,  but  you  can't  crash  an  airplane.  With 
aircraft  the  envelope  must  be  expanded  gradually. 

(Kemp.  1988) 

2.  Currently  all  DOD  programs  are  required  to  conduct 
both  development  and  operational  testing.  Development 
testing,  done  by  scientists  in  a  structured  lab 
environment  is  probably  similar  to  the  structured 
testing  conducted  for  Host.  Operational  testing  is 
normally  done  in  an  operational  environment,  therefore 
it  may  be  similar  to  the  unstructured  testing  conducted 
for  Host.  (Angeli,  1988) 

3.  DOD  needs  to  do  a  lot  more  unstructured  testing. 
(Christie,  1988) 

The  comments  addressing  element  A3  were  brief.  The 
phrases  'couldn't  agree  more'  (Qillogly,  1988),  and 
'definitely  needed,  and  done  in  DOD'  (Angeli,  1988) 
epitomized  the  respondent's  comments.  The  following 
statements  by  Colonel  Qillogly  summarized  the  responses 
concerning  element  A4 :  'Training  is  almost  always  planned 
for,  yet  it  is  not  actually  done  too  often.  Training  before 
delivery  is  hard  to  achieve.  The  user  must  keep  the  old 
system  going  and  often  can't  afford  to  send  users  off  for 
training'  (Qillogly,  1988).  The  responses  to  element  A5 
were  capsulized  in  the  comment  by  Ms.  Darleen  Druyun  that 
this  element  was  a  standard  requirement. 

Concluding  Comments.  The  respondents  who  didn't  answer 
the  third  interview  question  for  each  element  tended  to 
answer  it  in  the  conclusion,  addressing  the  entire 
management  strategy  in  general.  Even  the  respondents  who 
did  provide  specific  answers  tended  to  revisit  this  question 
in  the  conclusion.  Therefore,  some  of  those  concluding 
comments  are  presented  here. 
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Brig  3an  John  Douglass  commented  that  all  of  the 

elements  in  the  hypothesized  management  strategy  had  limited 

application.  His  statements  are  stunmarized  below: 

These  are  all  good  things  to  do,  especially  if  you 
are  buying  a  data  processing  system.  This  lends  itself 
well  to  communications  and  computer  systems  type 
architectures.  In  general  these  things  are  good,  but 
they  are  not  applicable  to  the  entire  scope  of 
acviuisition  in  the  military  today.  For  example,  in 
automated  data  processing  maybe  50  percent  of  the 
resources  go  into  software  assurances,  with  a  new 
aircraft  the  percentage  is  much  lower.  The  type  of 
tests  being  run  are  also  much  different.  (Douglass, 
1988) . 

General  Douglass'  cosunents  were  evaluated  against  the 
comments  of  the  other  respondents.  The  other  respondents 
believed  that  the  elements  found  to  be  necessary  in  the 
hypothesized  aianagenMnt  strategy  were  currently  used  in  DOD 
and  were  applicable  to  the  entire  scope  of  DOD  programs. 

The  following  statements  epitomize  the  comments  of  most  of 
the  respondents: 

1.  Everything  listed  here  is  important.  DOD  program 
managers  should  be  attuned  to  these  elements  and 
usually  are.  (Angeli,  1988) 

2.  All  of  the  itesis  listed  here  apply  to  all  DOD 
acquisition  prograsts .  They  are  all  common  sense. 
(Druyun,  1988) 

3.  There  are  things  that  the  Air  Force  has  dona  along 
these  lines,  and  those  things  are  good,  but  there  is 
still  a  long  way  to  go.  (Huegel,  1988) 

4.  DOD  currently  does  something  in  every  category 
here.  What  is  done  depends  on  the  program  phase. 

(Rak,  1988) 

When  evaluated  against  the  comments  of  the  majority  of  the 
respondents,  the  contention  that  these  elements  had  a 
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limited  applicability  was  refuted.  In  addition,  it  was 
countered  that  these  elements  specified  neither  the  types  of 
tests  to  be  run,  nor  the  percentage  of  resources  to  be 
devoted  to  specific  elements.  If  these  had  been  specified, 
then  the  applicability  of  the  elements  would  have  bean 
limited.  In  response  to  investigative  question  5c,  all  of 
the  elements  determined  to  be  necessary  for  program  success 
in  response  to  investigative  question  5b  were  found  to  be 
currently  used  in  DOD. .  Because  all  of  the  elements  found  to 
be  necessary  for  program  success  were  currently  used  in  DOD 
the  prerequisite  characteristics  of  the  elements  to  be 
evaluated  by  investigative  questions  5d  and  5e  were  not  met. 
Therefore,  both  investigative  questions  5d  and  5e  were  not 
addressed.  In  response  to  investigative  question  5f ,  all  of 
the  elements  determined  to  be  necessary  for  program  success 
were  found  to  be  applicable  to  all  programs. 

Elements  Suggested  for  Addition  to  the  Management  Strategy 
While  evaluating  the  elements  in  the  hypothesized 
management  strategy  the  respondents  identified  elements 
which  were  missing  from  the  strategy  that  they  believed  were 
necessary  for  program  success.  The  following  statement  by 
Colonel  (Ret)  Lindenfelser  provided  additional  Justification 
for  evaluating  the  respondents  recommendations:  *A  primary 
problem  with  this  list  of  elements  identified  by  the  Host 
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program  management  team  is  these  elements  can  all  be  found 
in  unsuccessful  programs  too.  A  successful  program  has  all 
of  these  menu  items  plus  a  few  additional  items,  especially 
stability*  (Lindenf elser ,  1988).  The  respondents’ 
recommendations  are  presented  and  analyzed  in  the  following 
three  sections.  The  sections  correspond  to  the  three 
management  strategy  categories  from  which  the 
recoBimendations  were  generated.  These  three  categories  are, 
Contract .  Teamwork .  and  Testing  and  Training. 

Contract .  As  pointed  out  by  Colonel  (Ret) 

Lindenf elser ,  a  primary  concern  of  the  respondents  was 
stability.  When  addressing  element  C4  Ms.  Druyun  pointed 
out  that  the  allocation  and  timing  of  program  funds  were 
crucial  to  program  execution,  but  these  were  not  possible 
without  stability.  Colonel  Tourino  also  commented,  'What  is 
fundamental  to  successful  major  systems  acquisition  is 
budget  stability*  (Tourino.  1988).  Colonel  (Ret) 
Llndenfelser  identified  the  three  types  of  stability  a 
program  must  have:  1)  requirements  stability,  2)  funding 
stability,  and  3)  environment  stability.  The  need  for 
requirements  stability  has  already  been  addressed  in  the 
Focus  category.  According  to  Colonel  (Ret)  Llndenfelser, 
funding  stability  starts  at  the  service  level,  and  it  must 
include  the  Office  of  the  Secretary  of  Defense  and  Congress. 
He  further  explained  that  environment  stability  was  a  result 
of  support  through  the  system  to  keep  the  program  alive. 
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including  Congressional  support.  Colonel  Angel i  also 
addressed  the  need  for  Congressional  support,  and  he 
presented  a  few  ways  of  increasing  that  support. 

Mr.  Arthur  Simolunas.  the  Host  Contracting  Officer 
Technical  Representative,  was  asked  if  Host  had  funding 
stability.  He  stated  that  the  Federal  Aviation 
Administration  had  eight  billion  dollars  set  aside  in  a 
tru'^t  fund  for  the  National  Aerospace  System  Plan  (NASP) 
improvement,  which  Host  fell  under.  Mr.  Simolunas  stated 
that  the  funding  stability  was  very  important  to  Host 
success.  This  response  indicated  that  Colonel  (Bet) 

Lindenf elser ' s  suggestions  for  maintaining  budget  stability 
required  exploration. 

Colonel  (Ret)  Lindenfelser  believed  that  it  was  crucial 
for  program  managers  to  understand  the  resource  allocation 
process.  He  stated,  ’Without  this  understanding  the  program 
manager  is  lost.  It  is  within  this  process  that  the  program 
manager  can  work  to  maintain  funding  stability.  The  program 
manager  has  little  control  over  output,  but  input  can  be 
provided'  (Lindenfelser.  1988).  Therefore  it  was  found  that 
an  element  requiring  funding  stability  should  be  added. 

This  element  should  require  the  program  manager  to 
understand  and  provide  inputs  to  the  resource  allocation 
process.  To  further  improve  the  budgeting  process  it  was 
also  found  that  Mr.  Christie’s  suggested  addition,  that  fund 
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requirements  be  known  and  budgeted  for  up  front,  be  added 
also . 

Mr.  Simolunas  was  asked  if  Host  had  environment 
stability.  To  that  he  replied  that  Congress  did  not  get  as 
involved  Department  of  Transportation  programs  as  with 
Department  of  Defense  programs.  He  emphasized  that  HASP 
improvement  was  viewed  as  vital  and  necessary  by  Congress. 
This  lent  additional  support  to  the  need  for  Colonel 
Angeli’s  suggested  method  for  increasing  environment 
stability . 

In  his  concluding  comments  Colonel  Angeli  stated,  ‘One 

major  element  that  is  missing  is  the  need  to  get 

congressional  support.  The  program  manager  must  take  action 

to  ensure  that  Congress  understands  the  program.  This  will 

help  to  maintain  budget  stability,  which  helps  to  maintain 

schedule  stability  which  both  make  the  program  manager’s  job 

much  easier*  (Angeli,  1988).  Before  outlining  his  strategy. 

Colonel  Angeli  cautioned  that  program  managers  must  always 

remember  to  go  through  the  appropriate  chain  of  command,  and 

that  they  are  not  allowed  to  lobby  Congress.  The  actions, 

recommended  by  Colonel  Angeli,  that  a  program  manager  can 

take  to  obtain  congressional  support  are  outlined  below: 

1.  Ensure  that  the  Legislative  Affairs  (LA)  section  in 
the  appropriate  Service  Secretary’s  Office  is  kept  well 
informed.  Program  managers  can  send  program  updates 
and  reports  to  the  LA  section  and  request  that  they  be 
passed  on  to  the  appropriate  congressional  offices  and 
committees . 
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2.  The  program  manager  can  arrange  to  visit  with  the 
appropriate  congressional  staff  member  or  professional 
military  assistant  from  the  House  Armed  Services 
Committee  through  the  LA  section.  The  program  manager 
can  use  this  visit  to  ensure  that  Congress  is  informed 
about  the  program  and  its  importance. 

3.  The  program  manager  can  request  to  support  the 
program  during  the  congressional  budget  hearings. 

Based  on  the  comments  by  Mr.  Simolunas,  and  using 

Colonel  Angeli’s  recommended  actions  as  a  guide,  it  was 

found  that  an  element  directing  program  managers  to  keep 

Congress  informed  through  official  service  liaisons  should 

be  added  to  the  management  strategy.  It  was  also  determined 

that  the  three  new  elements  recommended  for  addition  did  not 

apply  to  any  of  the  categories  already  existing.  Therefore 

a  new  category  was  identified  that  would  encompass  the  basic 

concepts  of  these  three  suggestions.  This  catego^'y  will  be 

named  Program  Stability. 

It  should  also  be  noted  that  Mr.  Theodore  Beckloff, 
Director,  Air  Traffic  Plans  and  Requirements  Service,  stated 
that  budget  stability  and  Congressional  support  were 
important,  but  without  the  other  elements  the  Host  program 
would  have  ’fallen  on  its  nose'. 

Te.3.mwork .  In  addressing  the  elements  in  teamwork  the 
respondents  were  concerned  that  the  Host  definition  of  team 
was  too  restrictive.  The  following  statement  by  Colonel 
(Ret)  Lindenfelser  emphasized  these  concerns:  'In  DOD  the 
definition  of  teamwork  is  very  broad.  It  includes  the 
efforts  of  the  system  program  office,  training  command,  Air 


Staff,  the  Secretariat,  the  Office  of  the  Secretary  of 


Defense,  Congress,  the  program  manager,  the  contractor,  and 
the  user'  (Lindenf elser ,  1986).  The  elements  suggested  for 
addition  in  this  area  revolved  around  the  expanded  team 
def inition . 

Colonel  (Ret)  Lindenfelser  stated,  'In  addition  to  the 
other  Teamwork  elements,  the  program  manager  must  understand 
the  contractor,  their  monitoring  system,  and  what  is  being 
done  in  engineering.  Any  successful  program  requires  this' 
(Lindenfelser,  1988).  This  suggestion  was  evaluated  with 
the  elements  already  recommended  in  the  Teamwork  category. 
The  evaluation  indicated  that,  if  properly  followed,  these 
elements  would  require  the  program  manager  to  understand  the 
contractor's  systems  to  the  extent  necessary  to  manage  the 
program.  Therefore  this  suggestion  was  recommended  for 
evaluation  in  subsequent  research,  and  not  included  at  this 
time . 

When  addressing  element  T2  Mr.  Joseph  Drelicharz 
stated,  'Communication  must  be  vertical,  horizontal,  and 
include  the  outside  chain  of  command.  For  example,  if  you 
don't  have  a  feel  of  how  the  local  paper  and  Congress  feel 
about  your  program  you  may  be  in  deep  trouble'  (Drelicharz, 
1988).  A  similar  suggestion  to  keep  Congress  informed  was 
already  approved  for  inclusion  in  the  management  strategy  in 
the  previous  section.  Therefore,  no  action  was  taken. 

Mr.  Drelicharz  also  noticed  a  shortfall  when  addressing 
element  T3 .  He  stated  that  it  was  vital  to  track  action 
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items  generated  in  meetings.  In  addition,  he  stated  that  it 
was  also  vital  to  have  an  agenda  prepared  by  the  program 
manager  for  each  meeting.  He  emphasized  that  a  frequent 
error  in  the  government  was  to  let  the  working  level  or  the 
contractor  prepare  the  agenda.  According  to  Mr.  Drelicharz, 
when  the  program  manager  prepares  the  agenda,  the  items 
addressed  and  the  information  provided  will  be  what  the 
program  manager  needs  and  thinks  are  important.  Colonel 
Tourino  also  identified  these  same  shortages.  He  stated, 
'This  element  CT3]  is  only  good  if  people  are  prepared  for 
the  meeting,  and  if  action  items  are  tracked*  (Tourino, 
1988).  In  analyzing  these  suggestions  the  data  collected 
during  the  exploration  of  Host  was  reviewed.  It  was  found 
that  action  items  were  visibly  tracked  and  presented  at 
every  meeting  observed.  It  was  also  noted  that  action  items 
were  entered  into  the  problem  database,  which  the  majority 
of  the  Host  respondents  cited  as  vital  to  Host  success. 
Finally,  Milton  Garwood,  Host  Computer  System  Planning  and 
Implementation  Specialist,  stated  that  an  agenda  was 
prepared  for  all  meetings.  However,  there  was  no  evidence 
to  support  the  suggestion  that  agendas  should  be  prepared  by 
program  managers.  Therefore  it  was  found  that  these  two 
suggestions  should  be  added  to  the  Teamwork  category:  1) 
Track  action  items  generated  in  meetings,  and  2)  Prepare 
meeting  agendas. 
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T*stinig  and  Training.  There  was  only  one  suggested 
addition  to  this  category.  While  addressing  element  A4  Mr. 
Drellcharz  stated,  *Yes,  but  this  Includes  much  more  than 
having  personnel  trained.  The  site  must  also  be  prepared 
and  ready  to  support  the  system  before  It  Is  delivered* 
(Drellcharz,  1988).  Based  on  the  interviews  with  the  Host 
hardware  implementation  team,  and  on  the  data  collected 
while  observing  site  preparations  It  was  found  that  this 
suggestion  should  be  added  to  the  Testing  and  Training 
category . 

Management  Strategy  For  Achieving  POD  Program  Success 

The  answers  to  the  investigative  questions,  and  the 
results  of  the  suggested  element  evaluations  provided  the 
Information  to  achieve  the  second  research  objective.  This 
achievement  is  presented  In  Tables  5.9,  and  5.10.  Table  5.9 
lists  the  first  three  categories  in  the  Management  Strategy 
For  Achieving  POD  Program  Success,  and  Table  5.10  lists  the 
last  four.  The  elements  listed  in  this  table  were  all  found 
to  be  necessary  for  program  success,  currently  used  in  POD, 
and  applicable  to  all  programs.  Those  elements  found  to 
require  modification  are  restated  as  revised.  The  elements 
recommended  for  addition  that  were  found  to  be  necessary  for 


program  success  are  also  listed. 


Table  5.9 


Management  Strategy  For  Achieving 
DOD  Program  Success  -  First  Section 


Program  Stability 

1.  Understand  and  provide  inputs  to  the  resource  allocation 
process 

2.  Fund  requirements  known  and  budgeted  for  up  front 

3.  Keep  Congress  informed  through  official  service  liaisons 


Contract 

1.  Award  Fee  and  Incentive  Fee  used  Jointly  (CPIF/AF) 

2.  Waterfall  implementation  plan 

3.  Realistic  schedule  providing  some  slack 

4.  Funds  budgeted  by  either  user  or  program  office  to 
support  participative  requirements 

5.  Requirements  generated  by  the  user  in  terms  of 
operational  need 

6.  Include  specific  contract  provisions  to  encourage 
contractor  cooperation 


Teamwork 

1.  All  constituents  involved  in  planning  and  scheduling 

2.  Both  lateral  and  vertical  communication  channels  open 

3.  Frequent  meetings  used  to  resolve  differences 

4.  Track  action  items  generated  in  meetings 

5.  Prepare  and  follow  meeting  agendas 

6.  One  face  to  users  and  management 

7.  All  functions  involved  in  problem  resolution 
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Table  5.10 


Management  Strategy  For  Achieving 
DOD  Program  Success  -  Second  Section 


Participative  Leadership 

1.  Site  representatives  involved  from  start  to  finish 

2.  User  input  requested  in  all  areas  and  all  phases 

3.  Site  input  actually  used 

4.  Users  kept  informed,  frequent  site  briefings  and  memos 

5.  Problems  solved  at  the  lowest  level  possible 

6.  Users  allowed  to  voice  disagreements  and  concerns 


Focus 

1.  No  bells,  whistles,  or  additional  fixes  were  added 

2.  Innovation  restricted  to  the  original  program  scope 

3.  Studies  performed  by  support  contractors  limited 


Progress  and  Performance 

1.  Actual  demonstration  of  requirement  before  sign>off 

2.  Technical  Performance  Measures  established  up  front 

3.  SMA  established  up  front,  and  achievable 

4.  All  problems  categorized  and  visible  in  INFO  database 

5.  Numerous  indicators  of  program  progress  used 


Testing  and  Training 

1.  Testing  as  early  in  development  as  possible 

2.  Both  structured  and  unstructured  testing  conducted 

3.  User  involved  in  planning  and  performing  test  scenarios 

4.  Site  personnel  trained  before  system  delivery 

5.  Site  prepared  and  ready  to  support  system  before- delivery 

6.  After  delivery  detailed  site  specific  testing  conducted 
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VI .  Conclusions  and  Recommendations 


Conclusion 

The  primary  assertion  of  this  thesis  was  that  the 
exploration  of  a  successful  program  would  reveal  management 
processes  that  facilitated  achieving  program  success  that 
were  not  previously  recommended  in  research  nor  used  in  DOD 
program  management.  The  exploratory  case  study  of  the 
Federal  Aviation  Administration’s  Host  computer  program 
resulted  in  a  management  strategy  hypothesized  to  help 
program  managers  achieve  program  success.  The  Host  program 
was  appropriate  because  it  met  all  of  the  criteria  used  to 
determine  program  success.  The  hypothesized  management 
strategy  was  a  composite  of  the  management  elements  and 
organizational  procedures  perceived  by  the  Host  program 
management  team  to  contribute  to  Host  success.  The  strategy 
was  comprehensive.  The  six  categories  encompassed  the 
following: 

1 .  Elements  that  should  be  included  in  the  contract  to 
facilitate  program  management. 

2.  Elements  consistent  with  current  management  theory 
to  ensure  the  cooperation  and  enthusiasm  of  the  program 
management  team,  the  user,  and  the  contractor. 

3.  Elements  to  keep  all  efforts  focused  towards 
program  goals,  and  to  track  progress  towards  those  goals. 

4.  Elements  to  ensure  that  the  end  product  is  what  was 
originally  specified  and  needed. 
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Despite  the  rigour  of  the  methodology  and  the 
comprehensiveness  of  the  resulting  hypothesized  management 
strategy  the  answer  to  investigative  question  5a  was  that  no 
new  concepts  had  been  discovered.  The  conclusion:  the 
primary  assertion  of  this  thesis  was  not  met  by  the 
exploration  of  the  Host  computer  program.  This  is  not  seen 
as  a  wasted  effort,  however,  as  it  tends  to  reassure  the 
reader  that  DOD  program  management  may  be  somewhat  better 
than  feared  or  than  implied  by  critical  assessments.  On  the 
contrary,  to  have  found  numerous  new  tools  of  successful 
programs  not  already  being  used  by  DOD  would  have  confirmed 
the  lack  of  successful  practices  in  DOD  program  management. 

Recommendations 

The  management  strategy  resulting  from  the  DOD 
acquisition  expert  survey,  conducted  to  evaluate  the 
hypothesized  strategy  formulated  during  the  Host  evaluation, 
was  found  to  be  a  good  guide  for  achieving  program  success. 
The  following  recommendations  were  based  on  the  findings  of 
the  expert  opinion  survey.  These  recommendations  are 
divided  into  the  seven  categories  contained  in  the 
management  strategy  that  resulted  from  the  beliefs  and 
recommendations  of  the  experts  interviewed. 

Reconunendation  No.  1.  Funding  stability,  identified  as 
crucial  to  program  execution,  is  primarily  a  function  of  the 
external  program  environment.  A  major  finding  of  the  expert 
opinion  survey  was  the  identification  of  actions  that  the 
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program  manager  could  take  to  increase  funding  stability. 

It  is  recommended  that  program  managers  take  the  following 
actions  to  increase  the  funding  stability  of  their  programs: 

1.  The  program  manager  must  learn  how  the  resource 
allocation  process  works,  and  provide  inputs  to  that 
process . 


2.  The  program  manager  must  expend  the  time  and 
resources  to  accurately  identify  funding  requirements. 

Those  requirements  should  include  consideration  for  the 
degree  of  program  risk. 

3.  The  program  manager  must  keep  Congress  informed  of 
the  potential  benefit  of  the  program  and  of  program 
accomplishments.  These  actions  can  be  accomplished  through 
letters,  reports  or  personal  visits  and  should  be 
coordinated  through  the  official  service  liaisons  to 
Congress . 

Recoounendation  Mo.  2.  The  management  of  a  program  can 
be  hindered  or  facilitated  by  the  structure  and  elements  of 
the  contract.  The  Host  program  management  team  members 
identified  six  contract  elements  that  facilitated  program 
management.  The  DOD  acquisition  experts  validated  the 
necessity  of  four  of  those  elements,  and  directed  the 
modification  of  the  other  two.  It  is  recommended  that  the 
program  manager,  working  in  unison  with  the  contracting 
officer,  incorporate  the  following  features  into  the 
contract  to  facilitate  the  management  of  the  program: 

1.  Use  award  fees  and  incentive  fees  jointly.  Match 
the  share  ratio  between  these  two  fees  to  the  degree  of 
program  risk.  Ensure  that  the  award  fee  percentage  is  at 
least  equal  to  the  incentive  fee  percentage. 

2.  Use  a  waterfall  implementation  plan  if  the  program 
involves  several  different  sites.  Apply  the  lessons  learned 
at  one  site  to  subsequent  sites. 
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3.  The  program  manager  must  expend  the  time  and 
resources  to  develop  a  realistic  schedule.  The  program 
manager  is  cautioned  not  to  be  optimistic.  The  schedule 
must  allow  a  degree  of  slack  commensurate  with  anticipated 
program  risk. 

4.  The  program  manager  must  ensure  that  funds  are 
budgeted  in  either  the  program  budget  or  in  the  user’s 
budget  to  support  participative  requirements.  It  is  also 
recoounended  that  DOD  investigate  routinely  including 
participative  requirements  in  the  program  budget. 

5.  The  program  manager  must  not  ’push"  requirements  or 
technology  on  the  user.  The  program  manager  must  ensure 
that  the  requirements  submitted  by  the  user  are  in  terms  of 
operational  need,  and  must  work  jointly  with  the  user  in 
making  the  tradeoffs  between  alternatives.  The  user  must  be 
involved  in  determining  if  the  specifications  selected  meet 
its  need. 

6.  The  program  manager  must  include  specific  contract 
provisions  to  encourage  contractor  cooperation.  The 
negative  reinforcement  of  holding  the  contractors  money  has 
a  significant  impact. 

Recommendation  Mo.  3.  The  factor  found  to  be  most 
important  for  successful  program  management  by  the  Defense 
Systems  Management  College  study  examined  in  Chapter  II  was 
a  teamwork  relationship  of  mutual  trust  between  government 
and  contractor  personnel  (Baumgartner,  1984:37).  Despite 
their  concerns  of  possible  conflict  of  interest  and  a 
historical  lack  of  cooperation  between  the  government  and 
the  contractor,  the  DOD  acquisition  experts  determined  that 
cooperation  was  necessary  for  program  success.  The 
importance  of  cooperation  and  communication  was  found  to 
extend  to  the  relationships  between  the  program  management 
team  members.  It  is  recommended  that  the  program  manager 
take  the  following  actions  to  increase  teamwork: 
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1.  After  the  basic  baseline  is  established,  the 
program  manager  must  get  all  constituents  involved  in 
planning  and  scheduling.  The  term  constituents  includes 
both  the  government  and  contractor  program  personnel.  This 
action  facilitates  the  need  to  dig  in  immediately  after  the 
contract  is  awarded  and  achieve  an  early  meeting  of  the 
minds.  All  constituents  learn  what  is  expected  from  them, 
and  what  the  program  goals  are. 

2.  The  program  manager  must  encourage  program  team 
members  to  communicate  across  functional  boundaries  and  to 
commiinicate  with  their  contractor  counterparts.  In  addition 
to  lateral  communication,  the  program  manager  must  emphasize 
the  need  for  team  members  to  keep  him/her  informed,  and  must 
provide  frequent  feedback  to  the  team  members  in  return. 

3.  In  addition  to  formal  meetings,  the  program  manager 
must  encourage  the  use  of  informal  meetings  to  resolve 
differences  between  the  government  and  the  contractor  as 
they  arise.  The  program  manager  must  also  require  that 
he/she  be  kept  in  the  loop.  This  is  in  line  with  the  idea 
of  a  team  effort. 

4.  The  program  manager  must  ensure  that  the  action 
items  generated  in  meetings  are  tracked.  This  will  increase 
the  productivity  of  the  meeting,  and  will  motivate 
participants  to  arrive  prepared. 

5.  The  program  manager  must  prepare  the  agenda  for 
meetings,  or  provide  substantial  input  to  its  preparation. 
This  ensures  that  the  information  provided  in  the  meeting  is 
what  the  program  manager  needs  and  believes  is  important. 

6.  The  program  manager  must  take  action  to  ensure  that 
one  face  is  presented  to  users  and  management,  and  must 
emphasize  the  importance  of  this  unity.  Presenting  one  face 
to  users  increases  user  confidence  and  morale.  Presenting 
one  face  to  management  increases  program  credibility.  The 
difficult  process  necessary  to  achieve  this  unity  clarifies 
program  direction  and  goals,  and  ensures  that  all  team 
members  are  on  the  right  track. 

7.  The  program  manager  must  work  with  the  contractor 
to  resolve  problems.  The  degree  of  cooperation  is  dependent 
on  the  terms  of  the  contract.  Cooperation  on  problem 
resolution  increases  the  chances  of  successful  program 
execution . 

Becommendation  No.  4.  The  heart  of  the  new  management 
theory,  credited  with  the  high  productivity  and  quality 
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levels  achieved  by  the  Japanese  and  select  U.S.  firms,  was 
user  involvement  and  participation.  The  beliefs  cf  the  DOD 
acquisition  experts  interviewed  were  corroborated  by  the 
findings  of  the  studies  evaluated  in  the  literature  review. 
It  was  found  that  participative  leadership  is  vital  to 
program  success.  The  following  actions  are  recommended  to 
ensure  user  involvement: 

1.  The  program  manager  must  involve  users  in  planning, 
scheduling,  and  tradeoff  evaluation  from  the  beginning  of 
the  acquisition  life  cycle  to  the  end. 

2.  The  program  manager  must  actively  seek  user  input 
in  all  phases  of  the  acquisition  life  cycle,  and  in  all 
operational  areas. 

3.  The  program  manager  must  use  the  input  collected. 

If  the  input  is  not  used  then  many  operational 
considerations  will  be  overlooked,  and  the  users  will 
quickly  stop  providing  feedback  and  suggestions.  Visible 
application  of  the  user's  input  will  increase  user 
enthusiasm  and  acceptance  of  the  system.  Enthusiasm  and 
acceptance  will  result  in  more  feedback  and  user  effort, 
which  will  contribute  to  achieving  program  success. 

4.  The  program  manager  must  keep  users  informed. 
Frequent  site  visits  will  provide  insight  to  user  attitudes 
and  potential  system  acceptance,  and  will  increase  user 
support  and  confidence.  The  program  manager  must  use  all 
means  of  communication  to  keep  the  user  updated  on  program 
progress,  and  to  educate  the  user  about  the  system  where 
necessary . 

5.  The  program  manager  should  delegate  authority  to 
the  program  management  team  members.  This  delegation  should 
include  an  emphasis  on  solving  problems  at  the  lowest  level 
possible.  This  trains  and  motivates  personnel,  while 
freeing  the  program  manager  to  tackle  the  major  problems. 

6.  The  program  manager  must  allow  users  and  program 
management  team  members  to  voice  disagreements  and  concerns. 
This  is  a  good  early  warning  device  for  potential  problems. 

Recommendation  No.  5.  Requirements  stability  was 
identified  by  the  DOD  acquisition  experts  as  DOD's  biggest 
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problem  and  was  identified  as  a  primary  factor  for  achieving 
program  success.  The  need  to  focus  program  efforts  was 
found  to  be  vital,  but  also  hard  to  accomplish.  The 
following  actions  are  recommended  to  help  the  program 
manager  with  this  goal: 

1.  The  program  manager  must  evaluate  each  potential 
addition  to  the  program  to  ensure  that  it  is  necessary  for 
achieving  program  goals  and  not  just  nice  to  have. 

2.  The  program  manager  must  take  care  to  ensure  that 
innovations  do  not  violate  the  program  baseline  or  schedule. 

3.  The  program  manager  must  give  support  contractors 
sufficient  guidance  and  tell  them  which  areas  to  focus  on  in 
their  studies.  Additional  studies  can  significantly 
increase  program  cost.  The  program  manager  must  be  tough  on 
these  additional  studies,  allowing  only  those  that  make  a 
contribution  to  the  program. 

Recommendation  No.  6.  An  important  factor  in  meeting 
program  schedule  and  in  gaining  user  acceptance  of  the 
delivered  system  is  monitoring  program  progress  and 
performance.  The  DOD  acquisition  experts  validated  the 
necessity  of  these  five  elements  identified  by  Host  program 
management  team  members  as  necessary  for  program  success. 

The  following  actions  are  recommended  to  facilitate  the 
monitoring  of  program  progress  an  performance: 

1.  The  program  manager  must  ensure  that  the  delivered 
system  meets  the  contract  specifications.  It  is  recommended 
that  government  program  team  members  be  delegated 
responsibility  for  specific  groups  of  requirements  and  held 
accountable  for  ensuring  that  their  assigned  requirements 
are  met. 

2.  The  program  manager  must  establish  the  technical 
performance  measures,  against  which  progress  and  performance 
will  be  assessed,  up  front.  It  is  recommended  that 
technical  performance  measures  be  tied  to  cost  performance 
measures  when  they  are  being  established. 
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3.  The  program  manager  must  also  establish 
repairabi 1 i ty ,  maintainability,  and  availability  (RMA) 
standards  up  front.  The  first  step  is  to  use  special 
studies  and  analysis  personnel  to  establish  the  minimum  RMA 
numbers  required  to  support  the  user’s  need.  The  second 
step  is  for  the  program  manager  to  get  user  approval  on  the 
minimum  RMAs ,  and  to  ensure  that  the  numbers  are  reasonable 
and  achievable.  The  concept  of  achievable  must  be 
emphasized . 

4.  The  program  manager  must  ensure  that  problems  are 
recorded,  categorized,  and  visible.  It  is  important  to 
record  problems  in  both  factory  and  operational  testing,  and 
to  record  a  problem  the  first  time  that  it  appears.  The 
program  manager  should  allow  only  the  problem  originator  to 
categorized  the  problem  as  resolved.  These  actions  will 
verify  that  the  problems  identified  are  fixed,  and  will 
ensure  maximum  lead  time  for  problem  resolution. 

5.  The  program  manager  must  collect  data  and  perform 
analysis  in  parallel  to  what  the  contractor  is  doing.  These 
prudent  management  actions  will  highlight  inconsistencies 
and  the  areas  where  questions  should  be  asked.  In  addition, 
this  is  necessary  because  the  system  program  office  deals 
different  problems  and  needs  different  data  than  the 
contractor  does. 

Recommendation  No.  7.  The  need  for  prototyping  and 
testing  was  identified  to  be  a  feature  of  most  successful 
programs  by  the  President’s  Blue  Ribbon  Commission 
(President’s  Commission,  1986b: 12).  The  Commission  noted 
that  prototyping  and  testing  would  contribute  materially  to 
improving  cost  and  schedule  estimates.  The  need  for  testing 
and  training  was  also  identified  by  the  Host  program 
management  team  and  was  verified  as  necessary  during  the 
expert  opinion  survey.  It  is  recommended  that  the  program 
manager  incorporate  the  following  elements  into  the  program 
implementation  plan: 

1.  The  program  manager  must  begin  testing  as  early  in 
development  as  possible.  Modular  testing  should  be  started 
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as  soon  as  possible  to  provide  maximum  lead-time  on  any 
problems  discovered. 

2.  The  program  manager  must  ensure  that  both 
development  and  operational  testing  are  conducted. 
Operational  testing  should  begin  early  in  advanced 
development.  Both  of  these  types  of  testing  are  necessary. 

3.  The  program  manager  must  involve  the  user  in  both 
planning  and  performing  test  scenarios.  User  input  is  vital 
to  ensure  that  an  operational  consideration  is  not 
overlooked . 

4.  The  program  manager  must  ensure  that  site  personnel 
are  trained  bef_rj  system  delivery.  A  system  is  worthless 
if  there  ar^  no  trained  personnel  to  use  it.  Program 
managers  are  encouraged  to  strongly  encourage  users  to  send 
their  personnel  for  training.  If  the  user  is  unable  to  send 
personnel  away  for  training  the  program  manager  should 
consider  sending  someone  out  to  the  site  to  conduct 
training . 

5.  The  program  manager  must  ensure  that  the  site  is 
prepared  and  ready  to  support  the  system  before  delivery. 

The  actions  necessary  to  accomplish  this  should  be  outlined 
in  programs  plan  and  schedule.  This  is  vital. 

6.  The  program  manager  must  conduct  detailed  site 
specific  testing  after  the  system  is  delivered  to  ensure 
that  the  system  performs  as  required  despite  site 
peculiarities . 


Areas  For  Further  Research 

During  the  course  of  this  study  two  primary  areas  were 
identified  for  further  research.  These  two  areas  are 
discussed  below. 

1 .  The  exploratory  research  methods  used  in  this  study 
investigated  a  successful  acquisition  program  to  develop  a 
management  strategy  that  was  hypothesized  to  help  program 
managers  achieve  program  success.  In  the  next  step  the 
hypothesized  management  strategy  was  evaluated  by  experts. 
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The  result  was  a  management  strategy  that  experts  believed 
would  help  DOD  program  managers  achieve  program  success.  I 
is  still  left  to  provide  conclusive  evidence  as  to  which 


elements  cause  acquisition  success.  Future  research  should 
be  conducted  to  provide  conclusive  evidence. 

2.  During  the  expert  interviews  several  respondents 
commented  that  the  principles  of  management  for  program 
management  were  well  known  throughout  DOD  and  that  the  real 
problem  was  in  implementation.  Future  research  should  be 
conducted  to  develop  recommendations  to  facilitate 
implementation  of  the  management  strategy  resulting  from 
this  research  effort. 
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Appendix  A:  Host  Computer  Program  Interviewees 


FAA  Headquarters.  Service  Level 


Beckloff,  Ted  B.  ATB-100,  Air  Traffic  Plans  and 
Requirements  Service  Director.  22  July  1988. 

FAA  Headquarters,  Division  Level 

Perie,  Michael  E.  AAP-200 ,  Manager,  System  Development 
Division,  28  September  and  22  December  1987. 

FAA  Headquarters.  Branch  Level 

Fleming,  Timothy  G.  AAT-120,  Chief,  Information  Systems 

Branch,  Automation  and  Software  Division,  23  December 
1987. 

Garwood,  Gail  E.  ATR-150,  Host  Computer  System  Planning  and 
Implementation  Specialist,  Air  Traffic  Advanced 
Automation  System  Requirements  Branch,  System  Plans  and 
Programs  Division,  6  August  1987  through  22  July  1988. 

Garwood.  Milton  D.  ATB-150,  Host  Computer  System  Planning 
and  Implementation  Specialist,  Air  Traffic  Advanced 
Automation  System  Requirements  Branch,  System  Plans  and 
Programs  Division,  6  August  1987  through  22  July  1988. 

Leabo ,  Don.  AAP-210,  Host  Computer  System  Hardware/Software 
Engineer,  Advanced  Automation  Program  Office,  Host 
Computer  System  Branch,  Systems  Development  Division, 

16  September  1987  through  22  July  1988. 

Manning,  Lewis  L.  ATR-210,  Host  Computer  System  Automation 
Specialist,  Air  Traffic  Advanced  Automation  Software 
Requirements  Branch,  System  Plans  and  Programs 
Division , 

25  September  1987. 

Marek,  Richard.  APM-240,  Program  Manager,  Host  Systems 

-  Implementation  Branch,  Air  Traffic  Control  Automation 
Division,  28  September  1987  and  29  January  1988. 

Martin,  Preston.  AAP-220,  Computer  Systems  Analyst,  Area 
Control  Computer  Complex  Program  Branch,  Air  Traffic 
Control  Advanced  Automation  Division,  22  December  1987. 
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Peters,  Bonald  M.  AAP-210,  Host  Computer  System 

Hardware/Sof tware  Engineer,  Advanced  Automation  Program 
Office,  Host  Computer  System  Branch,  Systems 
Development  Division,  16  September  1987. 

Ravenscroft,  Diane.  APM-240,  Program  Analyst,  Host  Systems 
Implementation  Branch,  Air  Traffic  Control  Advanced 
Automation  Division,  28  September  1987. 

Rymond ,  Mike.  AAP-210,  Host  Computer  System  Hardware/ 

Software  Engineer,  Advanced  Automation  Program  Office, 
Host  Computer  System  Branch,  Systems  Development 
Division,  23  December  1987. 

Sanford,  Bennie  L.  APM-240,  General  Mechanical  Engineer, 

Host  Systems  Implementation  Branch,  Air  Traffic  Control 
Automation  Division,  28  September  1987  and  22  July 
1988. 

Simolunas,  Arthur  A.  AAP-210,  Contracting  Officer  Technical 
Representative  and  Manager,  Host  Computer  System 
Branch,  Advanced  Automation  Program  Office,  Systems 
Development  Division,  22  July  1988. 

Strand,  Linda  R.  Contracting  Officer,  Advanced  Automation 
Branch,  Contracts  Division,  23  December  1987. 

Workman,  Carroll.  APM-240,  General  Mechanical  Engineer, 

Host  Systems  Implementation  Branch,  Air  Traffic  Control 
Automation  Division,  28  September  1987  and  22  July 
1988. 

Young,  James  R.  APM-240,  System  Implementation  Specialist, 
Host  Implementation  Branch,  Air  Traffic  Control 
Automation  Division,  28  September  1987. 

IBM,  National  Service  Division  Level 

Morris,  William  J.  Host  Program  Service  Executive,  FAA 
Project  Office,  National  Service  Division,  Atlantic 
City  NJ ,  15  and  16  September  1987. 

IBM,  Federal  Systems  Division  Level 

Beeson,  Don.  Manager,  Host  Computer  System  Field 

Deployment,  FAA  Project  Office,  Atlantic  City  NJ ,  15  & 
16  September  1987. 

Devlin,  Frank.  Manager,  Host  Computer  System  Physical 

Planning,  FAA  Project  Office,  Atlantic  City  NJ ,  15  &  16 
September  1987. 
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Svralgard,  Doug.  Host  Computer  System  Repairabil ity 

Maintainability  and  Availability  Manager.  Host  Systems 
Engineering,  Rockville  MD .  25  September  1987. 

FAA  Regional  Level 

Holtz,  Barton.  AQL-435,  Supervisor,  Automation  Engineering, 
Great  Lakes  Region,  Des  Plaines  IL,  16  September  1987. 

Lichlyter,  James.  AGL-550,  Air  Traffic  Host  Computer 

Program  Regional  Representative,  Enroute  Automation, 
Plans  and  Programs  Division.  Great  Lakes  Region,  Des 
Plaines  IL,  15  September  1987  through  29  January  1988. 

Porter,  John.  AGL-421,  Advanced  Automation  System 

Coordinator,  Regional  Operational  Airway  Facilities 
Division.  Great  Lakes  Region,  Des  Plaines  IL.  15  and  16 
September  1987. 

Syed .  Rizvi  A.  AGL-435 ,  Electronics  Engineer,  Automation 
Engineering,  Great  Lakes  Region,  Des  Plaines  IL,  16 
September  1987. 

Martin  Marietta  Regional  Level 

Brown,  Keith.  Regional  Integration  Representative, 
Indianapolis,  16  September  1987. 

FAA  Air  Route  Traffic  Control  Center  Level 

Bowman,  Andrew.  Central  Computer  Complex  Technician, 

Central  Computer  Complex,  Airways  Facilities  Sector, 
Indianapolis,  16  September  1987. 

Burks,  Thomas  M.  Assistant  Manager.  Airway  Facilities 
Sector,  Indianapolis,  16  September  1987. 
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Appendix  B:  Host  Computer  Proftram.  Eventa  Observed 


Second  Site  Survey  -  15  &  16  September  1987 
Indianapolis  Air  Route  Traffic  Control  Center 


EVENT  1:  FAA  Host  Computer  System  Information  Briefing 

PURPOSE:  To  provide  general  information  to  the  users 

located  at  the  Indianapolis  ARTCC .  The  following 
items  were  covered:  1)  Host  benefits,  2)  contract 
administration.  3)  FAA  Headquarters  program 
management  team  points  of  contact,  and  4)  the 
user’s  role  in  Host  implementation. 

PRESENTERS:  Milton  Qarwood,  Host  Computer  System  Planning 

and  Implementation  Specialist 
Don  Leabo ,  Host  Computer  System 
Hardware/Software  Engineer 

PARTICIPANTS:  Local  Integration  Contractor 

IBM  Nation  Service  Division 
IBM  Federal  Service  Division 
FAA  Headquarters  Program  Management  Team 
FAA  Regional  Air  Traffic 
FAA  Regional  Airway  Facilities 
Indianapolis  ARTCC  Air  Traffic 
Indianapolis  ARTCC  Airway  Facilities 


EVENT  2:  Informal  Meeting 

PURPOSE:  To  resolve  site  implementation  problems. 

PARTICIPANTS:  Air  Traffic  Program  Management  Team 

Specialist,  FAA  Headquarters 
Air  Traffic  Host  Computer  Program  Regional 
Representative 

Airway  Facilities  Advanced  Automation  System 
Coordinator 


EVENT  3:  IBM  Host  Computer  System  Deployment  and 
Maintenance  Briefing 

PDBPOSE:  To  present  the  findings  of  the  walk-thru 

assessment  of  site  preparations  for  Host  hardware 
delivery.  The  following  items  were  presented:  1) 
delivery  schedule,  2)  installation  schedule.  3) 
testing  procedures,  4)  action  item  review  and 
additions,  and  5)  support  requirements,  the  user’s 
role. 

PARTICIPANTS :  The  same  as  those  for  Event  1. 


Informal.  Impromptu  Meeting  -  25  September  1987 
FAA  Headquarters .  Washington  DC 


PURPOSE:  To  discuss  the  proposed  change  in  ARTCC  computer 

technician  staffing.  Alternatives  to  counter  the 
negative  effects  of  the  change  were  addressed. 

PARTICIPANTS:  Gail  Garwood.  Host  Computer  Planning  and 

Implementation  Specialist 
Lewis  Manning.  Host  Computer  System 
Automation  Specialist 


Additional  Host  Tasks  Meeting  -  25  September  1987 
IBM  Headquarters.  Rockville  MD 


PURPOSE:  To  plan  for  the  contract  conclusion,  and  to  assess 

areas  to  apply  excess  program  funds. 

PARTICIPANTS:  FAA  Headquarters  Program  Office 

FAA  Contracting  Officer  Technical 
Representative 
IBM  Program  Manager 

IBM  Federal  Systems  Division  Representatives 
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Weekly  Problem  Reaolution  Telephone  Conference 
28  September  1987 
FAA  Heedouartere .  Waahin^ton  DC 


PUBPOSE:  To  dlscuee  «ny  ppobleaie  relating  to  the  Host 

computer  system.  The  following  items  are 
addressed:  1)  actions  being  taken  to  solve 
problems,  2)  problem  status,  and  3)  comments  and 
questions. 

PASTiCIPAHTS :  All  ttranty  ARTCCs  represented 

All  Regions  represented 
FAA  Technical  Center 
FAA  Academy 

FAA  Headquarters  Program  Office 
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Appendix  C:  POD  Expert  Interview 
Recrueat  for  Interview  Letter 


Name  Date 

Position 

Address 

Dear  Name 

The  Air  Force  Institute  of  Technology  conducts  resident 
graduate  educational  and  research  programs  in  the  areas  of 
acquisition,  logistics  and  management.  As  part  of  the 
program  requirements,  students  conduct  research  on  a 
question  of  interest  to  a  specific  service,  or  to  the  DOD  in 
whole.  The  research  findings  are  subsequently  reported  In  a 
thesis . 

One  of  our  students,  1st  Lt  Barbara  J.  Cohen,  is  conducting 
a  study  in  the  area  of  program  management.  Using  a 
methodology  similar  to  that  of  the  Packard  Commission,  she 
is  attempting  to  identify  which  principles  used  in  non-DOD 
program  management  might  prove  useful  in  DOD  program 
management.  Her  rigorous  investigation  of  one  particularly 
successful  program  managed  by  the  Federal  Aviation 
Administration  (FAA)  indicates  that  the  FAA  program  applied 
some  innovative  techniques  which  might  be  applicable  within 
DOD.  Part  of  the  methodology  we  are  using  calls  for 
acquisition  experts  to  assist  in  evaluating  these  'lessons 
learned*  and  their  potential  applicability.  We  would  like 
you  to  be  one  of  these  experts. 

Lieutenant  Cohen  would  greatly  appreciate  the  opportunity  to 
interview  you  briefly  for  this  research.  Due  to  academic 
(class)  constraints,  she  will  have  only  one  two-week  period 
for  conducting  these  interviews.  Could  you  possibly  spare 
time  for  a  short  (30-45  minutes)  interview  during  the  last 
week  of  June  of  the  first  week  of  July? 

Please  have  your  secretary  advise  Lieutenant  Cohen  (Autovon 
705-4149  or  513-255-4149)  as  to  when  an  interview  can  be 
scheduled  during  this  period.  Her  research  will  be  greatly 
enhanced  by  your  contribution.  Thank  you  in  advance  for 
your  participation. 


CARY  L.  DELANEY 

Chief,  Defense  Resources  Division 
System  Acquisition  Management  Dept 
School  of  Systems  and  Logistics 
Air  Force  Institute  of  Technology 


1  Atch 

Host  Program  Fact 
Sheet 


Appendix  D:  POD  Expert  Interview 
Host  Pro><ram  Fact  Sheet 


The  Host  Computer  System  developed  by  IBM  is  one  of  the 
first  installments  of  the  FAA  multi-billion  dollar  plan  to 
modernize  the  National  Airspace  System  (NAS).  Host  will 
serve  as  the  backbone  processor  for  the  new  Initial  Sector 
Suite  System  until  replacement  by  Advanced  Automation 
Processors  in  the  mid-1900s.  IBM  won  this  4197  million 
contract  in  July  1985. 

Host  replaces  the  20-year-old  IBM  9020  computers 
currently  in  the  nation’s  20  Air  Route  Traffic  Control 
Centers  (ARTCC) .  The  new  IBM  mainframe  computers  are  ten 
times  faster  and  have  five  times  the  storage  capacity.  This 
replacement  involves  minimal  changes  to  existing  National 
Airspace  System  (NAS)  software.  Total  NAS  software  has  more 
than  two  million  lines  of  code,  and  Host  has  130,000  lines 
of  new  or  modified  NAS  code.  According  to  FAA  Administrator 
Donald  D.  Engen ,  '[Host!  will  allow  the  air  traffic  control 
system  to  keep  pace  with  projected  traffic  growth  over  the 
next  decade  and  accomodate  the  introduction  of  new 
automation  functions  that  will  both  enhance  safety  and 
increase  controller  productivity.*  Host  features  include 
increased  capacity,  improved  reliability,  faster  response 
time,  and  greater  on-line  availability. 

According  to  James  Q.  Cain,  Deputy  Director  of  the 
Advanced  Automation  Program  Office,  ’the  Host  program  has 
been  one  of  the  most  successful  technical  efforts  in  FAA 
history.  Beginning  with  the  first  delivery  to  the  Seattle 
center  in  November  1906,  the  FAA-IBM  team  has  met  or 
surpassed  every  major  program  milestone.’ 

Evidence  of  Host  Success 

1.  Every  control  center  has  accepted  the  system.  This 
is  a  direct  measure  of  user  confidence.  In  the  past  some 
systems  have  been  sent  back.  With  Host,  sites  have  been 
requesting  permission  to  drop  some  preliminary  tests  and 
accept  early. 

2.  Every  control  center  is  using  the  system  after 
acceptance.  It  is  up  to  a  site  manager  whether  to  use  a  new 
system  or  not.  There  are  currently  some  devices  on  location 
that  site  managers  refuse  to  use. 

3.  Host  is  ahead  of,  or  meeting  the  implementation 
schedule  in  all  areas. 

4.  All  reliability,  maintainability  and  availability 
(RMA)  and  technical  performance  measurements  (TPM)  are  met 
or  exceeded.  This  is  documented  in  test  results. 

5.  Control  centers  have  requested  that  the  old  system 
be  disconnected  60  days  early. 

6.  Host  came  in  under  budget. 
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Appendix  E:  POD  Expert  Interview 
Interview  Seesion  Orftanization 


REVIEW  PURPOSE  OF  INTERVIEW 

Purpose . 

Sponsor . 

Methodology,  similar  to  the  Packard  Commission. 

Studied  the  program  management  techniques  of  a  successful 
non-DOD  program  to  see  if  the  techniques/pr inciples  they 
used  might  apply  to  POD  program  management.  Studied  the 
Host  Computer  System  program  managed  by  the  FAA. 

If  you  are  willing,  I'd  like  to  share  with  you  some  of 
the  reasons  why  the  Host  program  management  team  perceived 
that  they  were  successful ,  and  use  your  expert  perception  to 
determine  if  there  is  any  thing  new  or  potentially 
applicable  to  the  way  we  manage  POP  programs. 

Clarify  if  necessary.  Asking  to  bounce  ideas  off  of 
the  expert,  not  asking  them  to  be  creative. 

PROVIPEP  AMY  BACKGROUWP  WANTEP/NEEPEP 

REVIEW  5  MAIM  CATEGORIES  FOR  HOST  SUCCESS 

Hand  respondent  the  'Reasons  for  Host  Success."  (The 
‘Reasons  for  Host  Success'  given  to  each  respondent  is 
contained  in  Appendix  F.) 

The  reasons  given  for  the  Host  program  success  fell 
into  five  major  areas.  This  paper  shows  those  five  areas 
and  the  supporting  reasons  within  each. 

I’ll  give  you  a  few  minutes  to  review  this  list.  After 
you  have  completed  your  review,  I  will  answer  any  questions 
you  may  have,  discuss  each  area,  and  ask  for  your  opinion  as 
to  the  usefulness/applicability  of  each  element. 

INTROPUCTION  FOR  EACH  CATEGORY 

Contract . 

Host  acquisition  success  began  when  an  environment 
conducive  to  program  management  was  established  in  the 
contract  and  statement  of  work  (SOW) . 

Teamwork . 

Everyone  interviewed  cited  teamwork  as  a  major  element 
in  the  success  of  the  Host  program.  The  Host  team  included 
the  FAA  program  management  team  members,  and  their  IBM 
counterparts.  Included  in  the  teamwork  category  is 
sufficient  authority  and  freedom  given  to  each  team  member 
to  carry  out  assigned  responsibilities. 


Participative  Leaderahlp. 

Everyone  interviewed  believed  that  it  is  necessary  to 
involve  the  user.  To  bring  in  the  people  who  need  and  will 
use  the  system.  The  field  personnel  were  treated  as 
experts,  and  their  expertise  was  used.  This  focus  on  the 
needs  of  the  user  increased  user  morale  and  resulted  in 
enthusiastic  user  support,  effort,  and  suggestions. 

Focus . 

The  Advanced  Automation  Division  manager  and  system 
engineers  emphasized  and  reemphasized  the  need  to  'rein 
people  in.'  They  cited  'focus*  as  a  primary  reason  for  Host 
coming  in  ahead  of  schedule  and  under  budget.  This  focus 
was  also  believed  to  contribute  to  teamwork  because  team 
members  could  focus  on  the  program  instead  of  continually 
focusing  on  changes. 

Contractor  and  System  Monitoring. 

This  category  was  viewed  as  vital  to  ensuring  that  Host 
remained  'on  track.'  This  was  partially  accomplished  by 
listing  the  primary  contractor  program  management 
requirements,  and  formal/informal  meeting  support 
requirements  in  the  SOW.  In  addition,  the  FAA  team 
carefully  monitored  program  progress  using  indicators  which 
were  independent  of  the  contractor's.  Attention  to  detail 
was  a  key  concept. 

Testing  and  Training. 

All  of  the  other  elements  must  be  in  line  for 
successful  testing  and  training.  Teamwork  between  the  FAA 
program  management  team  and  their  IBM  counterparts  was 
believed  to  be  vital  to  productive  testing.  The  testing 
requirements  outlined  in  the  contract  must  be  flexible 
because  there  is  insufficient  information  when  the  contract 
is  drafted  to  determine  all  of  the  testing  that  must  be 
done.  Flexibility  allows  specific  testing  requirements  to 
change  as  additional  information  is  gathered. 

Participative  leadership  with  the  resulting  user 
enthusiasm,  confidence  and  trust  was  viewed  as  vital  to 
successful  training.  Users  must  be  involved  in  testing  and 
training  if  they  are  to  be  confident  in  the  system,  and  have 
the  confidence  in  themselves  to  use  the  system  once  it  is 
installed.  All  interviewees  believed  that  these  elements 
were  vital  to  achieving  the  final  measures  of  success,  1. 
user  acceptance  and  2.  actual  use  of  the  new  system. 

QUESTIONS  TO  ASK  FOR  EACH  ELEMENT 

1.  Is  this  element  new  or  unique  to  DOD  program  management? 
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2.  Is  this  element  necessary  for  an  acquisition  program  to 
be  successful? 

2a.  If  no,  ask  ‘Why?*  DONE,  go  to  next  element. 

2b.  If  yes,  continue  with  questions. 

3.  Is  this  element  currently  used  or  potentially  applicable 
to  the  DOD  acquisition  process? 

3a.  If  no,  ask  'Why?*  DONE,  go  to  next  element. 

3b.  If  currently  used,  DONE,  go  to  next  element. 

3c.  If  potentially  applicable,  ask  'What  is  the  range 
of  programs  that  this  element  is  applicable  to?*  Continue 
with  questions. 

4.  What  must  be  done  before  this  element  can  be  implemented? 

VERIFY/FILL  IN  NOTES  TAKEN 

Clarify  notes  with  respondent. 

Check  that  main  points  were  noted. 

Verify  bibliography  information. 

FOLLOW-UP  &  USE  OF  INTERVIEW 


Inform  respondent  that  a  paraphrase  of  the  interview 
will  be  sent  in  5  to  7  days  for  him/her  to  review  and 
annotate  as  necessary. 

Ask  respondent  for  permission  to  include  the  annotated 
paraphrase  in  an  appendix  to  the  thesis,  and  to  draw  quotes 
from  it  for  the  analysis  of  the  interviews. 

THANK  THEM 
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Appendix  F:  POD  Expert  Interview 
Reasons  For  Host  Success 


Contract : 

Award  Fee  and  Incentive  Fee  used  jointly  (CPIF/AF) 

Waterfall  implementation  plan 

Realistic  schedule  providing  some  slack 

Funds  budgeted  to  support  participative  requirements 

Detailed  requirements  generated  by  the  user 

Problem  resolution  impacts  contractor  payment  schedule 

Teamwork : 

All  constituents  involved  in  planning  and  scheduling 

Lateral  communication  channels  open 

Frequent  meetings  used  to  resolve  differences 

Responsibilities  accepted,  not  assigned 

One  face  to  users  and  management 

All  functions  involved  in  problem  resolution 

Participative  Leadership: 

Site  representatives  involved  from  start  to  finish 
User  input  requested  in  all  areas  and  all  phases 
Site  input  actually  used 

Users  kept  informed,  frequent  site  briefings  and  memos 
Problems  solved  at  the  lowest  level  possible 
Users  allowed  to  voice  disagreements  and  concerns 

Focus : 

No  bells,  whistles,  or  additional  fixes  were  added 
Innovation  restricted  to  the  original  program  scope 
Studies  performed  by  support  contractors  limited 

Contractor  and  System  Monitoring: 

Actual  demonstration  of  requirement  before  sign-off 
Technical  Performance  Measures  established  up  front 
RMA  established  up  front,  and  achievable 
All  problems  categorized  and  visible  in  INFO  database 
Numerous  indicators  of  program  progress  used 

Testing  and  Training: 

Testing  as  early  in  development  as  possible 

Both  structured  and  unstructured  testing  conducted 

User  involved  in  planning  and  performing  test  scenarios 

Site  personnel  trained  before  system  delivery 

After  delivery  detailed  site  specific  testing  conducted 


Appendix  Q:  POD  Expert  Interview 
Thank  You,  Request  for  Review  Letter 


Name  Date 

Position 

Address 


Dear  Name 

Thank  you  for  your  time  and  for  the  opportunity  to  interview 
you.  Your  perceptions  and  comments  were  very  helpful. 

I  would  like  your  permission  to  include  a  brief  outline  of 
the  interview  in  an  appendix  to  my  thesis.  I  am  enclosing  a 
paraphrase  of  your  comments  for  each  of  the  program 
management  categories  which  we  discussed.  This  is  a 
preliminary  draft,  please  feel  free  to  make  additional 
comments  or  clarifying  adjustments  directly  on  it.  The 
final  appendix  inclusion  will  be  more  concise  than  this 
draft,  and  will  reflect  all  changes. 

Included  for  your  reference  are  the  Host  Fact  Sheet,  and  the 
Reasons  For  Host  Success  as  perceived  by  the  Host  program 
management  team. 

Please  return  the  annotated  draft  in  the  enclosed  envelope. 
If  no  adjustments  are  required  then  no  further  action  is 
necessary,  although  I  would  appreciate  a  quick  note  stating 
such.  Again,  thank  you  for  your  interest  in  this  project. 


BARBARA  J.  COHEN,  ILt,  USAF 
Graduate  Student 
School  of  Systems  and  Logistics 
Air  Force  Institute  of  Technology 
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1 .  Interview  Paraphrase 

2.  Host  Fact  Sheet 

3.  Host  Menu  For  Success 

4.  Address 


I 


! 
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Appendix  H:  POD  Expert  Interviews 
Paraphrase  of  Each  Interview 


NAME:  Lt  Col  Robert  P..  Angel i  (USA) 

TITLE:  Course  Director  for  Policy 

Defense  Systems  Management  College 
DSMC/SE-P 

Ft  Belvoir,  VA  22060-5426 

DATE:  7  July  88 

TIME:  1000 

PLACE:  DSMC,  Ft  Belvoir 


CONTRACT 

Award  Fee  and  Incentive  Fee  used  jointly  (CPIF/AF) 


Waterfall  implementation  plan. 

Realistic  schedule  providing  some  slack. 


Funds  budgeted  to  support  participative  reouirements . 


The  user  pays  their  own  TDYs  in  DOD.  To  put  TDY  funds  to 
support  user  participation  in  the  program  budget  would 
attract  some  attention. 

Detailed  requirements  generated  by  the  user.  I  have 
always  felt  that  the  requirements  came  from  the  user  in  the 
form  of  a  statement  of  need. 

Problem  resolution  impacts  contractor  payment  schedule. 


TEAMWORK 

Teamwork  makes  sense.  We  hope  in  DOD  that  these  good 
leadership  techniques  are  applied. 

All  constituents  involved  in  planning  and  scheduling. 
Yes.  Some  program  offices  may  not  have  time  to  do  this,  but 
this  is  the  better  way  of  doing  things. 

Lateral  communication  channels  open.  Yes,  this  is 
important . 

Frequent  meetings  used  to  resolve  differences.  Yes . 

It  is  especially  important  to  have  an  agenda  for  each 
meeting . 


168 


Responaibilities  accepted,  not  assigned.  The  less 
authoritative  method  of  trying  to  'sell'  the  task  may  be  a 
better  idea.  This  makes  a  lot  of  sense.  The  problem  is 
that  too  many  times  issues  are  to  'hot*  to  take  the  time 
necessary  to  do  this.  A  lot  of  folks  don’t  like  this  idea 
in  DOD ,  because  DOD  tends  to  be  a  lot  more  authoritative. 

The  concept  here  is  one  currently  being  taught  at  DSMC . 

One  face  to  users  and  manaj^ement.  This  is  a  good  idea, 
don’t  breach  the  ethical  bounds  though.  Danger,  maintain 
the  arm’s  length.  This  principle  should  be  applied  one  step 
further  to  include  speaking  with  one  voice  to  Congress. 

All  functions  involved  in  problem  resolution. 
PARTICIPATIVE  LEADERSHIP 


These  are  necessary.  Should  be  considered  beyond  the 
user  to  include  the  members  of  the  SPO.  These  are  all  just 
good  leadership  techniques,  especially  for  an  office 
environment . 

Site  representatives  involved  from  start  to  finish. 

User  input  requested  in  all  areas  and  all  phases. 

Site  input  actually  used. 

Users  kept  informed,  frequent  site  briefings  and  memos. 

Problems  solved  at  the  lowest  level  possible. 

Users  allowed  to  voice  disa>^reements  and  concerns. 


FOCUS 


All  of  these  are  important.  DOD’s  biggest  problem  is 
requirements  stability. 

No  bells,  whistles,  or  additional  fixas  were  added. 
Yes.  It  is  also  important  not  to  let  the  user  change  the 
basic  need. 

Innovation  restricted  to  the  ori^tinal  problem  scope. 

Yes  . 


Studies  performed  by  support  contractors  limited.  Yes . 

CONTRACTOR  AND  SYSTEM  MONITORING 

The  current  push  in  DOD  for  streamlining  doesn’t  really 
affect  any  of  these  items,  or  negate  the  need  for  them.  In 
DOD  the  streamlining  goals  include: 


169 


1.  Don’t  over-specify  requirements.  Specify  only 
those  requirements  that  meet  an  actual  need. 

2.  Shorten  the  chain  of  command  from  the  PM  to 
the  Secretary  of  Defense,  the  SAE. 

3.  Don’t  tell  the  contractor  how  they  should 
accomplish  the  specifications.  Let  the  contractor  make 
their  own  decisions  on  how  best  to  do  it. 

4.  Get  the  ‘boilerplate’  clauses  out  of  there. 
Start  at  ground  zero,  adding  only  the  clauses  that  are 
appropriate . 

Actual  demonstration  of  requirement  before  sign-off. 
This  sounds  reasonable.  DOD  is  currently  doing  this.  In 
the  Demonstration/Val idation  phase  there  is  a  new 
requirement  to  have  a  competitive  prototype  flyoff. 

Technical  Performance  Measures  established  up  front. 
Yes,  this  makes  sense. 

RMA  established  up  front,  and  achievable.  Yes,  this 
makes  sense.  In  DOD  the  acronym  is  RAM.  Achievable  is  a 
good  point.  It  is  important  for  the  PM  to  ensure  that  these 
numbers  are  achievable  before  they  are  put  down  in  the 
contract.  If  RAM  are  not  achievable,  then  the  PM  runs  into 
problems.  It  is  also  important  to  involve  the  user. 

Perhaps  DOD  has  to  focus  on  this  area  a  bit  more.  I  am 
not  sure  if  DOD  does  a  good  job  in  establishing  these. 

All  problems  categorized  and  visible  in  IMFO  database. 
This  should  be  done.  Maybe  we  aren’t  doing  this  in  DOD. 

You  wrould  need  to  talk  with  someone  from  ASD  to  be  sure. 

Numerous  indicators  of  program  progress  used.  Good . 
This  would  help  to  highlight  contradictory  information,  and 
potential  problems. 

TESTING  AND  TRAINING 


Testing  as  early  in  development  as  possible.  Yes.  If 
the  PM  can’t  test  a  full-up  system,  then  they  should  test 
the  components.  This  is  a  very  true  and  important  element. 
Development  testing  especially  needs  to  be  started  early. 
Although  DOD  should  do  this,  and  is  supposed  to  do  this  it 
is  frequently  not  done.  Both  PMs  and  users  are  afraid  of 
finding  problems,  and  tend  to  downplay  problems. 

Both  structured  and  unstructured  testing  conducted. 

This  is  an  interesting  point.  Currently  all  DOD  programs 
are  required  to  conduct  both  development  and  operational 
testing.  Development  testing,  done  by  scientists  in  a 
structured  lab  environment,  is  probably  similar  to  the 
structured  testing  conducted  for  Host.  Operational  testing 
is  normally  done  in  an  operational  environment,  therefore  it 
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may  be  similar  to  the  unstructured  testing  conducted  for 
Host.  In  operational  testing,  the  user  is  given  the  piece 
of  equipment  to  use.  This  normally  occurs  before  the 
Production/Deployment  phase. 

User  involved  in  planning  and  performing  test 
scenarios .  Definitely  needed,  and  done  in  DOD. 

Site  personnel  trained  before  system  delivery. 
Definitely  needed,  DOD  is  trying  to  do  this. 

After  delivery,  detailed  site  specific  testing 
conducted .  Yes,  this  is  necessary. 

CONCLUSIOM 

Everything  listed  here  is  important.  DOD  program 
managers  should  be  attuned  to  these  elements,  and  usually 
are.  The  primary  problem  is  that  these  elements  can  be 
difficult  to  follow  because  of  the  environment  surrounding 
DOD  acquisitions,  outside  of  the  program  manager’s  control. 
The  program  manager  must  always  be  balancing  cost,  schedule 
and  performance. 

One  major  element  that  is  missing  is  the  need  to  get 
congressional  support.  The  PM  must  take  action  to  ensure 
that  Congress  understands  the  program.  This  will  help  to 
maintain  budget  stability,  which  helps  to  maintain  schedule 
stability  which  both  make  the  PM's  management  job  much 
easier . 

A  word  of  caution:  DOD  personnel,  therefore  PMs  ,  are 
not  allowed  to  lobby  Congress.  The  PM  must  go  through  the 
Legislative  Affairs  (LA)  section  in  the  appropriate  service 
Secretary’s  office.  The  personnel  in  the  LA  section  are  the 
official  service  liaisons  to  Congress.  The  PM  must  always 
remember  to  go  through  the  appropriate  chain  of  command. 

Some  examples  of  actions  that  a  PM  can  take  to  obtain 
Congressional  support  follow: 

1.  Ensure  that  the  LA  section  is  kept  well 
informed.  The  PM  can  send  program  updates  and  reports  to 
the  LA  section  and  request  that  they  be  passed  on  to  the 
appropriate  Congressional  offices  and  committees. 

2.  The  PM  can  arrange  to  visit  with  the 
appropriate  Congressional  staff  member  or  professional 
military  assistant  from  the  House  Armed  Services  Committee 
through  the  LA  section.  The  PM  can  use  this  visit  to  ensure 
that  Congress  is  informed  about  the  program  and  its 
importance . 

3.  The  PM  can  request  to  support  the  program 
during  the  Congressional  budget  hearings. 


NAME:  Mr.  Thomas  P.  Christie 


TITLE:  Director 

Directorate  (Program  Integration) 

Under  Secretary  of  Defense  Acquisition 
The  Pentagon 
Washington,  D.C.  20330 

DATE:  6  July  88 

TIME:  1330 

PLACE:  The  Pentagon 


CONTRACT 


Award  Fee  and  Incentive  Fee  used  jointly  (CPIF/AF) . 

Waterfall  implementation  plan. 

Realistic  schedule  providing  some  slack.  Yes ,  a 
realistic  schedule  is  necessary. 

Funds  budgeted  to  support  participative  requirements. 
Yes,  but  more  is  required  in  addition  to  this.  It  is 
imperative  that  fund  requirements  are  known  and  budgeted  for 
up  front.  The  entire  budget  must  be  realistic. 

Detailed  requirements  >;enerated  by  the  user.  Too  often 
wo  are  reaching  for  the  moon  with  requirements  in  the  name 
of  the  user.  The  user  becomes  a  parrot.  What  is  needed  is 
some  realism  up  front.  What  is  needed  in  the  very  beginning 
are  requirements  that  are  technically  possible,  realistic, 
and  feasible. 

Problem  resolution  impacts  contractor  payment  schedule. 
TEAMWORK 


The  relationship  between  the  government  and  the 
contractor  has  been  damaged,  currently  each  side  views  the 
other  as  an  adversary.  The  government  must  stop  operating 
with  the  attitude  that  contractors  are  fleecing  us.  Efforts 
are  being  made  to  repair  the  damage.  We  are  trying  to 
change  government  attitudes,  to  consider  Industry  as  a 
partner.  But  the  damage  will  never  be  repaired  as  long  as 
Congress  and  the  Secretary  of  Defense  continue  'industry 
bashing . ' 

All  constituents  involved  in  planning  and  schedulinX. 
This  is  a  given. 

Lateral  communication  channels  open. 
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Frequent  meetings  used  to  resolve  differences. 


Responsibilities  accepted,  not  assigned. 

One  face  to  users  and  mana>{ement . 

All  functions  involved  in  problem  resolution. 
PARTICIPATIVE  LEADERSHIP 


Site  representatives  involved  from  start  to  finish. 
User  input  requested  in  all  areas  and  all  phases. 

Site  input  actually  used. 

Users  kept  informed,  frequent  site  briefings  and  memos 
Problems  solved  at  the  lowest  level  possible. 

Users  allowed  to  voice  disagreements  and  concerns. 

FOCUS 

Mo  bells,  whistles,  or  additional  fixes  were  added. 
Excellent,  this  is  absolutely  needed. 

Innovation  restricted  to  the  orijjinal  problem  scope. 
Yes,  this  is  important.  It  is  also  hard  to  do  because  so 
many  layers  can  interfere  with  the  program. 

Studies  performed  by  support  contractors  limited.  Yes 

CONTRACTOR  AND  SYSTEM  MONITORING 

The  best  thing  to  do  is  to  give  the  contractor  the 
requirement,  and  then  back  off.  Let  the  contractor  do 
whatever  is  necessary  to  achieve  that  requirement. 

The  government  needs  to  take  the  regulations  off  both 
the  PM's  back,  and  the  contractor’s  back.  Will  we  ever  see 
it?  The  Under  Secretary  of  Defense  Acquisition  is  trying, 
for  example  the  Enterprise  Program  was  established,  but 
there  have  been  many  setbacks. 

Actual  demonstration  of  requirement  before  sign-off. 

Technical  Performance  Measures  established  up  front. 

PMA  established  up  front,  and  achievable. 

All  problems  categorized  and  visible  in  INFO  database. 
For  Host  this  was  probably  necessary,  it  might  be  hard  to  d 
in  DOD .  In  addition,  the  SPO  and  the  user  often  cover  up 


problems  because  they  are  afraid  that  the  program  may  be 
cut,  or  because  the  contractor  gets  them  to  believe  that  the 
problem  can  and  will  be  easily  fixed.  This  is  a  very 
political  issue. 

Numerous  indicators  of  program  progress  used. 


TESTING  AND  TRAINING 

Testing  as  early  in  development  as  possible.  This  is  a 
very  important  point.  Too  often  the  program  is  a  long  way 
into  the  acquisition  life  cycle  before  there  is  any 
indication  of  actual  item  performance.  It  is  important  to 
find  problems  early,  before  the  program  gets  over  budget, 
and  while  there  is  sufficient  time  to  fix  them. 

Both  structured  and  unstructured  testing  conducted. 

Yes.  Unstructured  tests  are  very  important,  but  they  are 
not  often  done.  Without  unstructured  tests  a  program  often 
goes  into  production  with  "faulty"  software.  DOD  needs  to 
do  a  lot  more  unstructured  testing. 

User  involved  in  planning  and  performing  test 
scenarios .  Yes,  both  structured  and  unstructured  testing 
needs  to  be  done  in  the  hands  of  the  operators. 

Site  personnel  trained  before  system  delivery.  Yes . 

After  delivery,  detailed  site  specific  testing 
conducted .  Yes . 

CONCLUSION 

I  agree  that  all  of  these  areas  are  important.  All  of 
the  points  under  testing  are  necessary,  all  of  the  elements 
under  focus  are  very  important,  a  schedule  providing  some 
slack  is  vital,  and  it  is  also  important  to  make  sure  that 
fund  requirements  are  known  and  budgeted  for  up  front. 
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NAME:  Brig  Oen  John  W.  Douglass 

TITLE:  Director 

Directorate  of  Program  Planning  and  Integration 

SAF/AQX 

The  Pentagon 

Washington,  D.C.  20330 

DATE:  29  June  1988 

TIME:  1100 

PLACE:  Pentagon 


CONTRACT 


Award  Fee  and  Incentive  Fee  used  jointly  (CPIF/AF) . 


TEAMWORK 


This  is  a  lost  art  in  the  USAF  due  to  Fraud  Waste  & 
Abuse  witch-hunts.  Poor  publicity  leads  to  poor  public 
perception  of  the  government  and  contractors  alike.  DOD 
focuses  on  excessive  'tail  covering.’  All  of  these  factors 
contribute  to  the  broadening  gap. 

Every  day  the  Air  Force  and  the  contractor  get  further 
and  further  away  from  being  a  good  team.  Industry  does  not 
want  to  do  business  with  us  because  we  are  perceived  as  a 
miserable  customer.  Industry  is  also  concerned  with  how  the 
public  perceives  them.  This  area  is  of  critical  concern. 

The  trend  is  in  the  wrong  direction.  The  DOD  and 
contractors  are  being  pushed  apart  because  the  military  is 
under  assault.  The  Pentagon  is  trying  to  turn  this  around. 

All  constituents  involved  in  planning  and  scheduling. 


Lateral  communication  channels  open. 
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All  functions  involved  in  problem  resolution. 
PARTICIPATIVE  LEADERSHIP 

Site  representatives  involved  from  start  to  finish. 

User  input  requested  in  all  areas  and  all  phases. 

Site  input  actually  used. 

Uaera  kept  informed,  frequent  aite  briefings  and  memos. 
Problema  solved  at  the  lowest  level  poaalble. 

Uaera  allowed  to  voice  diaa^peementa  and  eoneerns . 


FOCUS 

Focus  is  needed.  You  must  be  tough  in  ferreting  out 
the  bells  and  whistles  from  the  program. 

Mo  bells,  whistles,  or  additional  fixes  were  added. 

Innovation  restricted  to  the  original  problem  scope. 

Studies  performed  by  support  contractors  limited. 
CONTRACTOR  AMD  SYSTEM  MONITORING 

There  is  the  dilemma  of  standardized  versus 
individually  tailored  contractor  monitoring  procedures.  In 
the  Defense  Enterprise  Program  each  program  is  being 
tailored  differently.  In  this  program  the  Air  Force  is 
stripping  the  regulatory  requirements  to  tailor  the  degree 
of  regulation  to  program  risk.  This  program  is  expected  to 
expand . 

The  problem,  if  everyone  is  tailoring,  how  do  you  apply 
a  regulation?  The  challenge  will  be  to  review  the 
individual  programs  under  Enterprise,  and  to  identify 
regulations  being  uniformly  eliminated. 

Another  new  program  is  Flexible  Oversight.  If  the 
contractor  has  a  plant  that  is  running  well  with  good 
quality  then  we  [the  government}  will  pull  out.  On  the 
other  hand,  if  performance  degradation  occurs  the  government 
will  crawl  in  with  the  contractor.  The  expected  result  is 
that  poor  contractors  will  fix  the  defects,  and  that  good 
contractors  will  have  reduced  cost,  possibly  higher  profit, 
because  less  oversight  is  required. 

The  bottom  line,  'You  have  to  take  risk."  You  can’t  be 
scared  away  from  contractors.  A  contractor  database  is  a 
noble  goal ,  and  possibly  a  good  system,  but  it  is  a  better 
system  to  take  drastic  action  against  any  contractor  you  see 
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Technical  Performance  Measures  established  up  front. 
RMA  eatabliahed  up  front,  and  achievable. 

All  problema  cate><orized  and  visible  in  IMFO  database. 
Numerous  indicatora  of  program  proHresa  used. 

TESTING  AND  TRAINING 

Testin><  as  early  in  development  as  possible. 

Both  structured  and  unstructured  teatin^t  conducted. 

User  involved  in  planning;  and  performing;  test 
scenarios . 

Site  personnel  trained  before  system  delivery. 

After  delivery,  detailed  site  specific  testing 
conducted . 

CONCLUSION 


These  are  all  good  things  to  do,  especially  if  you  are 
buying  a  data  processing  system.  This  lends  itself  well  to 
ESD,  EDP  thru  700  series  regs ,  and  communications  type 
acquisitions.  In  general  these  are  good,  but  they  are  not 
applicable  to  the  entire  scope  of  acquisition  in  the 
military  today. 

If  you  are  buying  a  new  aircraft  all  of  these  things 
are  important,  but  they  are  not  important  in  the  same  kinds 
of  ways.  For  example,  in  EDP  maybe  50%  of  the  resources  go 
into  software  assurances,  with  a  new  aircraft  the  percentage 
is  much  lower.  The  type  of  tests  being  run  are  also  much 
different.  The  environment  is  much  different.  The 
variability  of  human  error  has  a  different  impact. 

There  is  no  single  model  perfect  for  all  acquisitions. 
You  must  have  flexibility  to  match  the  acquisition  strategy 
to  the  specific  program.  Yet  you  must  not  do  unproductive 
things,  you  must  challenge  and  get  rid  of  unproductive 
regulations. 


! 
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NAME:  Mr.  Joseph  Drelicharz 


TITLE:  Professor  of  Engineering  Management 
Defense  Systems  Management  College 
DSMC/PMC-S 

Ft  Belvoir,  VA  22060-5426 

DATE:  6  July  88 

TIME:  1000 

PLACE:  DSMC,  Ft  Belvoir 


CONTRACT 


Award  Fee  and  Incentive  Fee  used  jointly  (CPIF/AF) .  In 
the  classroom  award  fees  are  being  touted,  and  instructors 
are  shying  away  from  incentive  fees.  Personally,  I  am  very 
afraid  of  award  fees  because  they  have  a  tendency  to  become 
personal  service  contracts. 

Waterfall  implementation  plan.  This  is  necessary,  and 
currently  a  standard  procedure.  It  is  the  normal  deployment 
plan  to  pass  on  lessons  learned. 

Realistic  schedule  providing  some  slack.  This  is 
necessary  and  true,  but  it  is  an  unrealistic  thing  to  do. 

It  is  unrealistic  because  managers  are  graded  by  schedule, 
causing  them  to  be  optimistic  and  go  with  a  tight  schedule. 
Even  if  a  manager  submits  a  realistic  schedule  to  begin 
with,  as  the  schedule  goes  up  the  chain  of  command  for 
approval  each  level  trims  off  its  percentage,  and  an  overly 
optimistic  schedule  is  still  the  result. 

For  a  program  manager,  meeting  schedule  is  first 
priority,  making  contract  coat  and  actual  performance  second 
priority.  Schedule  drives  everything. 

Funds  budgeted  to  support  participative  requirements. 
Disagree.  It  is  neither  wrong  nor  bad  for  users  to  fund 
their  own  TDYs  to  support  the  project.  After  all,  who  is 
the  project  for?  The  user’s  willingness  to  do  this  is  a 
good  measure  of  user  desire  and  interest.  A  prudent  PM 
would  also  have  funds  available  for  users  who  truly  can  not 
afford  to  fund  their  own  TDYs. 

Detailed  requirements  generated  by  the  user.  No , 
definitely  not.  The  user  does  not  necessarily  have  the 
knowledge  and  information  to  define  the  requirements  for 
high  technology  and  'black  box*  systems.  The  requirements 
generation  must  be  a  joint  process  between  the  SPO  and  the 
user  to  be  realistic. 

Problem  resolution  impacts  contractor  payment  schedule. 
Careful,  this  could  take  away  the  contractor’s  motivation. 
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From  the  contractor's  point  of  view,  "all  of  these 
safeguards!  Don’t  you  trust  me  I  Aren’t  we  a  team?" 

Perhaps  a  better  way  to  handle  this  is  to  include  it  in  the 
award  fee  somehow  instead  of  a  separate  element  in  the 
contract . 

TEAMWORK 

All  constituents  involved  in  planning  and  scheduling. 
Detailed  scheduling  and  planning  done  jointly  is  good.  But, 
the  basic  baseline  must  already  be  nailed  in  the  contract, 
and  serve  as  the  base  for  the  scneduling  and  planning. 

Lateral  communication  channels  open.  This  statement  is 
inadequate.  Communication  must  be  vertical,  lateral,  and 
include  the  outside  chain  of  command.  For  example,  if  you 
don’t  have  a  feel  of  how  the  local  paper  and  Congress  feel 
about  your  program  you  may  be  in  deep  trouble.  If  you  do, 
you  may  be  able  to  avoid  trouble. 

The  SPO  members  should  call  their  contractor 
counterparts  daily,  and  work  in  unison. 

Frequent  meetings  used  to  resolve  differences.  Maybe 
frequent  is  not  a  good  idea.  For  every  monthly  informal 
review  session  the  contractor  might  spend  three  weeks 
preparing,  and  he’s  going  to  use  his  best  people.  It  is 
vital  to  track  action  items  generated  in  the  meeting.  It  is 
also  vital  to  go  to  the  meeting  prepared.  A  frequent  error 
in  the  government  is  to  let  the  working  level  or  the 
contractor  prepare  the  agenda.  The  agenda  should  be 
prepared  by  the  PM  and  coordinated  with  the  contractor. 

When  the  PM  prepares  the  agenda,  the  items  addressed  and  the 
information  provided  are  what  the  PM  thinks  are  important 
and  needs . 

Responsibilities  accepted,  not  assigned.  The  Air  Force 
is  also  a  matrixed  operation.  The  Air  Force  must  do  both. 
Sometimes  it  is  better  to  assign  responsibilities,  and 
sometimes  it  isn't.  It  depends  on  the  individuals, 
projects,  position  in  the  acquisition  cycle,  etc. 

One  face  to  users  and  management.  Vital  to  program 
success.  In  DOD  this  is  a  tough  thing  to  do.  There  are  too 
many  ’special  interest’  groups  ’flying’  out  to  the 
contractor  and  giving  the  contractor  guidance.  The  program 
manager  must  be  continually  reminding  the  contractor  that 
these  special  interest  groups  can  not  change  the  agreement 
already  established  between  the  PM,  contracting  officer  and 
the  contractor. 

All  functions  involved  in  problem  resolution. 
Definitely.  The  contractor’s  problems  are  our  problems. 
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This  is  the  only  way  that  it  will  work.  But  remember,  both 
sides  must  be  compensated  for  their  contributions. 

PARTICIPATIVE  LEADERSHIP 


Some  participative  leadership  is  necessary,  but  these 
are  overstated. 

Site  representatives  involved  from  start  to  finish.  I 
don’t  want  the  user  involved  to  this  extent,  the  user  does 
not  have  the  necessary  knowledge  base.  The  statement  of 
need  must  be  generated  by  the  user,  but  the  PM  must  decide 
’how  to  get  there  from  here.'  I  recommend  that  the  user  be 
involved  only  in  operational  considerations,  need 
definition,  testing,  and  implementation. 

User  input  requested  in  all  areas  and  all  phases.  No  , 
not  in  all  areas,  only  in  operational  areas.  I  agree  that 
the  user  should  provide  input  in  all  phases  of  the  program 
acquisition  life  cycle. 

Site  input  actually  used.  Yes,  this  is  necessary  to 
the  best  extent  possible. 

Users  kept  informed,  frequent  site  briefings  and  memos. 
Yes,  as  much  as  the  user  cares  to  be  kept  Informed. 

Problems  solved  at  the  lowest  level  possible.  Of 
course  this  is  important. 

Users  allowed  to  voice  disagreements  and  concerns. 

Yes.  The  whole  problem  is  to  negotiate. 

FOCUS 


No  bells,  whistles,  or  additional  fixes  were  added. 
Agree,  as  long  as  this  does  not  mean  shutting  off  all  inputs 
to  requirements.  Everyone  gets  smarter  as  the  development 
process  progresses,  and  the  requirements  must  still  be  open 
to  these  inputs.  Of  course  the  inputs  and  changes  should 
narrow  down  the  closer  the  program  gets  to  production,  and 
the  baseline  should  get  stiffer. 

The  primary  problem  with  this  is  that  new  RDT&E 
programs  take  10  to  15  years  to  procure.  If  the  baceline  is 
firmed  up  too  early,  then  we  will  end  up  with  a  system 
composed  of  technology  that  is  20  years  old. 

Using  common  sense  is  the  best  guidance  for  this  area. 

Innovation  restricted  to  the  original  problem  scope. 
This  is  really  an  affordability  issue.  A  laudable  approach, 
but  the  scope  of  most  DOD  programs  will  cover  the  moon.  For 
example,  the  scope  of  some  programs  limits  the  system  to 
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‘air  breathing*  devices, 
bombers  and  missiles. 


"Air  breathing*  includes  fighters, 


Studies  performed  by  support  contractors  limited. 
Agree.  It  is  important  to  give  the  support  contractors 
sufficient  guidance,  and  to  tell  them  which  areas  to  focus 
on.  Studies  are  generally  most  appropriate  very  early  or 
very  late  in  the  program. 

CONTRACTOR  AND  SYSTEM  MONITORIMO 


Actual  demonstration  of  requirement  before  sign-off. 

You  have  to  look  at  the  whole  picture  when  determining  if  a 
requirement  has  been  met.  For  example,  if  an  aircraft  has  a 
requirement  to  be  able  to  travel  at  mach  .8  and  only 
achieves  .79  during  the  demonstration  do  you  sign  the 
requirement  off? 

Technical  Performance  Measures  established  up  front. 
This  is  a  good  thing  to  do,  but  is  also  very  hard  to  do. 

The  cost  to  achieve  the  desired  technical  performance 
measures  must  be  considered  before  they  are  established. 

They  must  also  be  tied  to  cost  performance  measures. 

RMA  established  up  front,  and  achievable.  Agree.  It 
is  important  to  run  studies  up  front  to  see  what  RMA  are 
necessary  for  the  item  to  be  effective.  The  main  point: 
usually  the  wrong  people  establish  RMAs .  RMAs  should  be 
established  by  special  studies  and  analysis  personnel  using 
wargames  models.  The  special  studies  should  be  used  to 
establish  the  minimum  R^  numbers  required  to  support  the 
user’s  need.  The  PM’ s  duty  is  to  get  user  approval  on  the 
minimum  RMAs,  and  to  ensure  that  the  numbers  are  reasonable 
and  achievable.  Often  the  RMAs  are  increased  above  the 
minimum  necessary  if  the  higher  numbers  are  achievable. 

All  problems  categorized  and  visible  in  INFO  database. 
Problem  categorization  is  an  age-old  problem.  This  is  a 
good  idea. 

Numerous  indicators  of  program  progress  used.  These 
are  necessary.  This  is  not  micro-managing,  nor  is  it  a 
matter  of  trusting  the  contractor.  It  is  simply  a  matter  of 
tracking.  I  want  to  see  this  in  the  program  office  and  in 
the  HQ  staff  office.  This  is  important  and  necessary 
because  the  SPO  and  HQ  deal  with  different  problems  as  does 
the  contractor. 

TESTING  AND  TRAINING 

Testing  as  early  in  development  as  possible.  As  a 
general  statement  this  is  correct  and  necessary,  but  you 
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muflt  be  careful  of  what  testing  means.  If  early  testing  is 
to  be  conducted  on  modules,  it  should  be  stated  as  such. 


Both  structured  and  unstructured  testing  conducted. 
Without  a  doubt  this  is  important.  Unstructured  tests  could 
be  conducted  during  operational  testing.  Currently  the  DOD 
conducts  two  siajor  types  of  structured  testing:  1) 
development  testing  to  ensure  that  the  product  meets 
specifications,  testing  the  product  against  the  environment 
defined  by  the  users  requirements,  2)  operational  testing  to 
ensure  that  the  product  meets  user  needs,  testing  the 
product  in  the  user’s  actual  environment. 

User  involved  in  planning  and  performing  test 
scenarios .  Yes.  user  involvement  is  definitely  needed  in 
planning  and  performing  operational  test  scenarios. 

Site  personnel  trained  before  system  delivery.  Yes  , 
but  this  includes  much  more  than  having  personnel  trained. 
The  site  must  also  be  prepared  and  ready  to  support  the 
system  before  it  is  delivered. 

After  delivery,  detailed  site  specific  testing 
conducted .  Yes,  this  is  needed  continuously  and  forever. 
Don't  forget  the  importance  of  test  feedback,  tracking 
action  items,  and  correcting  action  items.  A  big  feedback 
loop  is  needed. 

COWCLUSIOM 


The  views  expressed  herein  represent  Mr.  Drelicharz’s 
personnel  opinions  and  may  not  reflect  any  positions  held  or 
promulgated  by  DOD  or  DSMC. 
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NAME:  Ms.  Darleen  Druyun 


TITLE:  Principal  Assistant  to  the  Deputy  Chief  of  Staff 
for  Contracts 
AFSC/PK 

Andrews  AFB ,  D.C.  20334 

DATE:  27  June  88 

TIME:  1200  hr 

PLACE:  HQ  AFSC ,  Andrews 


CONTRACT 

It  is  important  to  match  the  contract  type  to  the 
inherent  program  risk.  If  this  is  not  done  then  the 
contractor  is  put  in  a  'mission  impossible*  situation. 
Program  risk  is  composed  of  technical >  schedule  and  cost 
risk,  all  of  which  must  be  balanced  and  offset  against  each 
other.  A  CPIF/AF  type  contract  indicates  that  the  contract 
was  high  risk. 

Award  Fee  and  Incentive  Fee  Used  Jointly.  The  Award 
Fee  is  very  important.  Currently  HQ  AFSC  is  preparing  an 
Award  Fee  Guide  to  increase  the  combined  use  of  AF  and  IF  in 
the  Air  Force.  The  AFSC  goal  is  to  're-energize'  the  use  of 
the  Award  Fee . 

Use  award  fee  to  focus  the  contractor's  attention  on 
that  which  is  important,  critical  to  program  success.  Have 
the  evaluation  criteria  change  to  meet  what  is  critical  to 
meet  each  milestone. 

Award  fees  are  necessary  because  they  motivate  the 
contractor  in  areas  other  than  cost-the  primary  purpose  of 
incentive  fees.  Some  examples  of  areas  award  fees  can  be 
used  to  motivate/focus  the  contractor: 

1)  put  the  right  talent  on  the  project, 

2}  to  get  personnel  up-to-speed  faster, 

3)  improve  the  management  information  systems  used, 

4)  improve  contractor’s  management  of  subcontractors, 

5)  improve  quality 

‘I  am  a  firm  believer  in  the  power  of  Award  Fees.*  They  are 
applicable  to  every  contract  because  they  are  built  around 
the  areas  that  the  PM  considers  critical  to  his/her  specific 
project.  Award  Fees  are  especially  Important  in  the  more 
complex  environment. 

To  be  effective,  the  potential  payoff  from  the  Award 
Fee  must  be  substantial.  The  percentage  should  be  equal  to 
that  for  the  incentive  fee. 

Contractor’s  like  Award  Fees  because  the  process 
provides  them  with  feedback.  They  have  a  more  concrete  idea 
of  where  they  stand.  Feedback  is  also  provided  to  the  PM. 
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But,  Award  Fees  are  not  used  much  because  they  are 
work.  Work  to  develop,  and  a  lot  of  work  to  administer. 

Waterfall  implementation  plan. 

Realistic  schedule  providing  some  slack.  According  to 
AFSC  Acquisition  Strategy  Principles,  one  of  the  criteria 
against  which  the  acquisition  strategy  is  to  be  judged  is 
the  degree  to  which  the  schedule  is  realistic.  It  is  stated 
that  the  program  schedule  must  be  realistic.  The  user  need 
date  must  be  balanced  against  the  resources  available. 

Funds  budgeted  to  support  participative  requirements. 
Important,  a  result  of  teamwork  between  SPO  functions. 
According  to  AFSC  Acquisition  Strategy  Principles,  the 
proper  allocation  and  timing  of  program  funds  are  crucial  to 
program  execution.  The  problem  is  the  unstabilizing  effect 
congress  has  on  the  funds  appropriated. 

Detailed  requirements  generated  by  the  user. 

Necessary,  again  part  of  teamwork.  According  to  AFSC 
Acquisition  Strategy  Principles,  before  a  program 
acquisition  strategy  can  properly  be  defined,  the  PM  and  the 
user  must  have  a  clear  understanding  of  what  is  required. 

The  acquisition  strategy  must  reflect  that  understanding. 

One  of  the  criteria  against  which  the  acquisition  strategy 
is  to  be  judged  is  the  degree  to  which  the  strategy  meets 
the  users  needs. 

Problem  resolution  impacts  contractor  payment  schedule. 
TEAMWORK 

AFSC  is  emphasizing  teamwork  internally  within  the  SPO. 
It  is  the  PM  that  ensures  that  all  constituents  work  as  a 
team.  The  SPO  organization  can  either  be  a  separate  team  or 
matrixed  in. 

Integrating  all  functions  is  vital.  The  PM  must  check 
and  crosscheck  how  the  requirements  and  responsibilities  of 
all  functions  interrelate  to  be  sure  that  one  area  is  not 
‘killing*  another. 

All  constituents  involved  in  planning  and  scheduling. 
Definitely. 

Lateral  communication  channels  open.  The  PM  sets  the 
stage.  Communication  up,  down  and  laterally  is  very 
important.  Caution,  the  PM  must  be  kept  informed  and  in  the 
picture,  the  right  hand  must  know  what  the  left  hand  is 
doing . 

Teamwork  is  vital  to  providing  a  systematic  process  to 
apply  AFSC  resources  to  the  users 's  need/problem.  All  of 
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th«  functional  disciplines  involved  must  function  as  an 
integrated  unit.  From  AFSC  Acquisition  Strategy  Principles. 


Frequent  meetings  used  to  resolve  differences. 

Responsibilities  accepted,  not  assigned.  Yes ,  to  some 
degree. 

One  face  to  users  and  management. 

All  functions  involved  in  problem  resolution.  Yes . 
PARTICIPATIVE  LEADERSHIP 

All  elements  listed  below  are  absolutely  necessary. 

'The  user  must  be  intimately  involved*.  If  all  of  the 
requirements  established  by  the  user  can’t  be  met,  then  it 
is  up  to  the  user  to  make  the  appropriate  tradeoffs.  It  is 
important  to  talk  with  the  user,  lay  out  options  as  well  as 
problems . 

Site  representatives  involved  from  start  to  finish. 

User  Involved  and  sometimes  assigned  full-time  to  the  SPO  in 
most  Air  Force  programs.  Often  users  even  go  visit  the 
contractor  with  the  SPO  team. 

User  input  requested  in  all  areas  and  all  phases. 
Extremely  important.  By  involving  the  user,  they  understand 
the  tradeoffs  that  have  been  made,  and  the  acceptance  rate 
is  higher.  Must  have  the  users  and  the  SPO  working  side-by- 
side  . 

With  the  ATF  TAC  users  are  assigned  full-time  to  SPO. 
The  decisions  concerning  requirements  tradeoffs  and 
alternatives  are  left  up  to  the  user  to  make. 

Site  input  actually  used.  Yes 

Users  kept  informed,  frequent  site  briefings  and  memos. 
Yes,  this  is  necessary. 

Problems  solved  at  the  lowest  level  possible.  Yes 

Users  allowed  to  voice  disagreements  and  concerns.  Yes 


FOCUS 

This  is  common  sense,  but  it  is  where  you  run  into  most 
of  the  problems.  A  golden  rule  is,  'Don’t  change  the 
baseline.'  But  if  participative  leadership  is  in  place  the 
user  won’t  want  to  change  the  baseline  because  they  will 
understand  the  alternatives,  and  will  have  made  all  of  the 
necessary  tradeoffs. 
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No  bella,  whistles .  or  additive  fixes  were  added. 


Innovation  reatricted  to  the  ori>tinal  projtram  scope. 

Studies  performed  by  aupport  contractors  limited. 
CONTRACTOR  AND  SYSTEM  MONITORING 

Excellent  MIS  on  contractor’s  part  can  negate  many  of 
these  requirements.  The  degree  of  system  monitoring 
necessary  depends  on  the  contractor’s  reputation,  their  MIS, 
and  the  degree  of  risk/type  of  contract.  In  some  cases 
'micromanagement*  may  be  necessary.  Sometimes  a  program  has 
to  managed  by  ‘ inchstones . * 

In  all  situations,  the  PM  must  gather  as  much  feedback 
and  data  as  possible.  The  PM  must  be  alert  and  'not  sleep' 
through  monthly  meetings. 

Actual  demonstration  of  requirement  before  sign-off. 

Technical  Performance  Measures  established  up  front. 

RMA  established  up  front,  and  achievable. 

All  problems  categorized  and  visible  in  INFO  database. 

Numerous  indicators  of  program  progress  used. 

TESTING  AND  TRAINING 

According  to  AFSC  Acquisition  Strategy  Principles,  a 
disciplined  test  process  must  be  defined  up  front.  The 
result  will  be  executable  test  objectives,  measurable  test 
criteria  and  increased  chance  of  program  success. 

Testing  as  early  in  development  as  possible.  This  is 
so  true.  Start  modular  testing  ASAP.  If  one  module 
collapses  the  redesign  of  all  modules  is  required,  the  more 
lead-time  on  the  problem  the  better.  Up  front  testing  can 
prevent  the  need  for  major  redesign. 

Both  structured  and  unstructured  testing  conducted. 

Ms.  Druyun  doesn’t  believe  that  the  Air  Force  uses  much 
unstructured  testing.  It  could  be  applicable. 

User  involved  in  planning  and  performing  test 
scenarios .  Yes,  this  is  necessary. 

Site  personnel  trained  before  system  delivery.  This 
should  be  a  PM  goal .  The  PM  should  make  every  attempt  to 
get  support  equipment  and  simulators  out  there  ahead  of 
time . 


c 
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Assured  System  Availability  is  a  new  AFSC  innovation  to 
get  support  equipment  and  documentation  into  the  field  ASAP. 
It  involves  writing  the  right  incentives  into  the  contract 
to  motivate  the  contractor.  A  new  concept. 

After  delivery,  detailed  site  specific  testing 
conducted :  A  standard  requirement. 


COHCLUSION 


All  of  the  items  listed  above  apply  to  all  DOD 
acquisition  programs.  They  are  all  common  sense.  The 
difficulty  lies  in  getting  them  implemented.  A  new  AFSC 
concept  should  help  do  this. 

The  Acquisition  Strategy  Panels  have  a  proactive  versus 
a  reactive  focus.  The  panel  was  established  to  ensure  that 
PMs  have  good  upfront  planning.  The  PM  plan  should  address 
the  implementation  of  each  element  in  these  'menus  for 
success . * 

The  panel  is  composed  of  experts  representing  each 
functional  area.  The  user  is  included  as  part  of  the  panel. 
The  ASP  will  make  sure  that  the  PM  is  on  the  right  track, 
and  provide  lead  time  ahead  of  problems.  AFSC  Regulation 
800.53  is  the  document  establishing  the  ASP. 

According  to  AFSC  Regulation  800-53:  'The  overall 
objective  of  an  ASP  is  to  ensure  that  a  systematic  and 
disciplined  approach  has  been  developed  to  meet  the  users’ 
needs  within  resource  constraints  and  that  the  approach  is 
consistent  with  the  program  management  directive.' 

The  ASP  is  to  be  conducted  before  the  acquisition  plan 
is  submitted  for  approval.  The  program  director  brings  in 
the  acquisition  strategy  that  they  have  formulated,  and 
briefs  that  strategy  to  the  ASP. 

Air  Force  Systems  Command  has  recently  outlined 
‘Acquisition  Strategy  Principles.'  The  acquisition  strategy 
is  defined  at  the  plan  for  program  execution,  the  PM’s 
blueprint  to  achieve  the  user’s  requirements.  The  document 
outlines  three  key  principles  to  be  used  as  a  framework  for 
designing  an  appropriate  acquisition  strategy:  1)  Resources, 
2}  Processes,  and  3)  Criteria. 

Resources  are  the  raw  materials  available  to  AFSC  such 
as  people,  technology,  dollars,  and  facilities. 

Process  are  provided  by  all  involved  functional 
disciplines  to  apply  the  available  resources  to  the  problem. 
The  processes  must  function  as  an  integrated  unit  and 
include : 

1)  Requirements 

2)  Engineering 

3)  Support 

4)  Test 

5)  Budget 

6)  Business 
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Criteria  are  used  to  assess  the  acquisition  strategy 
developed  so  far,  and  ensure  all  areas  are  addressed.  The 
following  questions  must  be  asked  of  every  acquisition 
strategy  to  ensure  its  excellence: 

1)  Does  it  meet  user's  need? 

2)  Has  the  appropriate  technology  been  applied? 

3)  Is  the  plan  integrated? 

4)  Have  alternatives  been  considered  throughout 

the  process? 

5)  Are  the  risks  understood? 

6)  Is  the  schedule  realistic? 

7)  Are  the  costs  understood? 

8)  Is  the  process  streamlined? 

9)  Is  the  plan  executable? 

10)  Does  it  represent  best  value? 


HAME:  Colonel  Harry  Oillogly  III 


TITLE:  Deputy  Chief  of  Staff.  Technology 
and  Bequirements  Planning 
AFSC/XT 

Andrews  AFB,  D.C.  20334 

DATE:  27  June  88 

TIME:  1000 

PLACE:  HQ  AFSC .  Andrews 


CONTRACT 

Award  Fee  and  Incentive  Fee  used  jointly  (CPIF/AF) . 
Standard . 

Waterfall  implementation  plan.  Useful,  have  used. 
Realistic  schedule  providing  some  slack.  Need . 

Funds  budgeted  to  support  participative  requirements. 

Need . 


Detailed  requirements  generated  by  the  user.  A 
potential  problem.  When  you  involve  a  variety  of  groups  in 
establishing  requirements,  each  group  will  bring  in 
different  functional  expectations.  Each  individual  also 
brings  biases  for  what  system  should  do. 

'How  did  they  stabilize  their  requirements?*  User 
stability  is  an  Important  ingredient.  There  is  a  big 
difference  between  FAA  mission  and  DOD  mission.  The  FAA 
mission  is  much  more  stable. 

Problem  resolution  impacts  contractor  payment  schedule. 
A  good  way  to  incentivise  the  contractor.  Negative 
reinforcement,  holding  contractor’s  money  has  a  big  impact. 

TEAMWORK 


Each  PM  has  their  own  management  philosophy.  Most  PMs 
in  military  rarely  have  black  and  white  decisions,  so  they 
must  rely  on  support  from  their  people  and  from  the 
contractor . 

The  PM  team  must  discover  alternatives  and  be  available 
to  discuss  alternatives.  The  PM  then  makes  decisions  based 
on  these  inputs. 

Teamwork  is  important  but  as  a  word  of  caution,  some 
displacement  is  needed  between  the  program  manager 
(including  the  program  management  team)  and  the  contractor. 
This  is  necessary  to  ensure  that  the  PM  won't  make  decisions 
conciliatory  to  contractor’s  whims. 
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Teamwork  is  good,  but  the  PM  still  has  to  keep  the 
contractor's  feet  to  the  fire. 

All  constituents  involved  in  planning  and  Scheduling- 

Lateral  communication  channels  open. 

Frequent  meetings  used  to  resolve  differences. 

Responsibilities  accepted,  not  assigned. 

One  face  to  users  and  management. 

All  functions  involved  in  problem  resolution. 
Fundamental.  It  is  especially  important  to  involve 
logistics  representatives. 

PARTICIPATIVE  LEADERSHIP 

All  of  these  items  are  absolutely  essential. 

Site  representatives  involved  from  start  to  finish. 
Baaed  on  WIS  experience,  this  is  very  important.  The  user 
must  help  -develop  requirements  and  schedules. 

User  input  requested  in  all  areas  and  all  phases.  Yes 

Site  input  actually  used.  Yes 

Users  kept  informed,  frequent  site  briefings  and  memos. 
Most  definitely.  If  users  are  not  kept  informed,  the 
program  manager  will  have  problems. 

Problems  solved  at  the  lowest  level  possible.  "I’ve 
always  believed  in  this." 

Users  allowed  to  voice  disagreements  and  concerns.  Yes 

FOCUS 


Very  important  area.  The  program  manager,  contractor 
and  user  must  all  come  to  an  agreement  as  to  what  is  going 
to  be  done  in  the  program.  You  can’t  let  the  baseline  be 
destroyed.  All  ideas  must  be  evaluated  against  the 
baseline.  Innovations  should  only  be  dismissed  if  they 
violate  your  baseline.  A  baseline  is  violated  if  the 
schedule  is  extended,  or  if  funds  aren’t  available. 

Mo  bells,  whistles,  or  additional  fixes  were  added. 

Innovation  restricted  to  the  original  program  scope. 

Studies  performed  by  support  contractors  limited. 
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CONTRACTOR  AMD  SYSTEM  MOMI TORINO 


Actual  demonatration  of  requirement  before  si>;n-off. 

In  a  C-cubed  procurement  demonstration  is  possible.  In 
other  procurements  it  isn’t.  Prototyping  is  nice  because  it 
reduces  risk.  Sometimes  *fly  before  you  buy'  does  not  work. 

Very  important.  Definitely  worth  your  time.  It  is  not 
necessarily  micromanagement  to  hold  individuals/the 
contractor  responsible.  In  WIS  every  requirement  was 
validated.  The  AF  currently  has  a  mechanism  to  do  this,  it 
is  called  the  'Critical  Design  Review. ' 

Technical  Performance  Measures  established  up  front. 
Very  important.  The  best  way  to  establish  TPMs,  is  to  get 
the  contractor  to  propose  what  they  are  going  to  do  to 
monitor  themselves.  This  is  especially  true  during 
production/development . 

RMA  established  up  front,  and  achievable.  RMA  mus t 
definitely  be  achievable  and  quantitative.  It  is  up  to  the 
program  office  to  analyze  if  user  specified  RMA  standards 
are  ‘real  life'  achievable.  The  PM  must  look  at  what  is 
'achievable*  versus  what  the  user  believes  ‘should  be.' 

The  PM  should  only  promise  what  good  analytical  checks 
of  the  state-of - the-art-techno logy  as  applied  to  the 
environment  of  intended  use  indicate  are  achievable.  This 
is  especially  crucial  when  procuring  aircraft,  because  the 
environment  is  unknown  and  many  judgement  calls  must  be 
made . 


All  problems  categorized  and  visible  in  INFO  database. 
This  is  necessary.  The  Air  Force  currently  uses  a  system 
similar  to  INFO  called  'FRACAS.*  It  is  important  to  record 
problems  in  BOTH  factory  and  operational  testing.  Often 
problems  are  not  recorded  in  factory  testing,  then  those 
same  problems  are  discovered  later  in  operational  testing. 
Earlier  recording  would  have  provided  extra  lead  time. 

Numerous  indicators  of  program  progress  used.  Depends 
on  the  system  being  procured,  and  the  type  of  contract.  In 
software  intensive  programs  close  monitoring  is  vital 
because  no  one  really  has  a  good  handle  for  how  long  it 
takes  to  write  so  many  lines  of  code.  For  software 
procurement,  the  use  of  ‘software  metrics'  is  recommended. 

TESTING  AND  TRAINING 


Testing  as  early  in  development  as  possible.  Yes 

Both  structured  and  unstructured  testing  conducted. 
There  are  possible  problems  with  this.  You  must  be  careful 
that  the  unstructured  testing  doesn’t  generate  new 
requirements  that  will  disrupt  the  baseline.  Unstructured 


tasting  is  fine  if  tfie  purpose  is  to  validate  a  specific  set 
of  problems. 

With  WIS,  they  were  concerned  about  achieving  a  user 
friendly  system,  so  unstructured  testing  could  have  been 
used  to  address  the  problem.  Potential  'ungraceful  exits' 
could  have  been  identified. 

A  final  caution,  if  unstructured  testing  is  used,  be 
sure  that  requirements  are  well  structured,  and  that  the 
baseline  is  firmly  established. 

User  involved  in  planning  and  performing  teat 
scenarios .  Couldn’t  agree  more. 

Site  personnel  trained  before  system  delivery. 

Training  is  almost  always  planned  for,  yet  it  is  not 
actually  done  too  often.  Training  before  delivery  is  hard 
to  achieve.  The  user  must  keep  the  old  system  going  and 
often  can’t  afford  to  send  users  off  for  training.  One 
possible  fix  is  to  send  trainers  into  the  field. 

After  delivery,  detailed  site  specific  testing 
conducted .  Yes,  this  is  necessary. 


COMCLUSIOM 


Looking  at  these  'Menus  For  Success',  all  of  these  are 
important.  The  principles  of  management  for  program 
management  are  pretty  well  known  throughout  DOD,  they  are 
taught  at  DSMC ,  and  they  are  outlined  in  regulations.  The 
problem  is  implementation. 

If  implementation  is  the  problem,  then  the  key  is 
writing  a  program  plan  ahead  of  time  telling  how  and  when 
you  plan  to  meet  all  of  the  items  on  this  'Menu  of  Success.' 
It  is  also  important  that  the  plan  address  all  of  the 
elements,  not  just  a  few. 
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NAME:  Lt  Col  Paul  Huegel 


TITLE:  Chief.  Progranus ,  Analysis  and  Initiatives  Division 
Directorate  of  Policy  &  Programs 
AFSC/SDX 

Andrews  AFB,  D.C.  20334 

DATE:  28  June  1988 

TIME:  1000 

PLACE:  HQ  AFSC .  Andrews 


CONTRACT 


All  of  these  elements  are  fundamental.  It  is  most 
important  to  correctly  define  user  requirements.  It  is  next 
most  important  to  set  up  the  contract  to  facilitate  program 
success.  The  Acquisition  Strategy  Panel  and  AFSC 
Acquisition  Strategy  Principles  help  to  accomplish  this. 

AFSC  is  very  proud  of  their  efforts  in  this  area.  The 
Acquisition  Strategy  Panel  facilitates  up  front  planning  for 
contract  type  and  program  management.  The  panel  is  composed 
of  a  ‘set'  group  of  experts,  all  hand  picked,  that  go  out  on 
reviews . 

Award  Fee  and  Incentive  Fee  used  jointly  (CPIF/AF) . 
Current  regulations  clearly  state  that  the  focus  on 
contracts  should  be  consistent  with  all  program 
characteristics,  including  risk.  (Cost  Plus  versus  Firm 
Fixed  Price  type  of  contracts)  The  combination  of  IF  and  AF 
is  an  interesting  concept.  The  Air  Force  hasn’t  done  much 
with  Award  Fees  to  motivate  the  contractor.  Darleen  Druyun 
is  the  contracting  expert  for  AFSC,  and  can  shed  some  light 
into  this  area. 

Waterfall  implementation  plan. 

Realistic  schedule  providing  some  slack. 

Funds  budgeted  to  support  participative  requirements. 
The  idea  here  is  good.  This  is  definitely  an  area  that 
requires  attention.  The  main  problem  is  that  application 
would  require  changing  some  regulations.  Air  Force  budget 
policy  states,  the  person  making  the  trip  will  fund  the  TDY. 
This  means  that  the  user  must  fund  their  own  trips  to 
provide  program  support. 

In  the  past  the  PM  had  enough  flexibility  within 
his/her  RDT&E  3600  money  to  enable  the  program  to  fund  many 
participative  user  TDYs .  The  days  of  sufficient  travel 
money  are  gone.  Prior  obligation  of  RDT&E  funds  to  other 
areas,  and  the  lack  of  slack  in  the  program  O&M  funds  have 
virtually  eliminated  the  PMs  ability  and  willingness  to  fund 
user  TDYs. 


Users  are  supposed  to  fund  their  TDYs  out  of  their  own 
O&M  money.  This  is  a  problem  because  there  is  rarely  any 
slack  in  organization  O&M  funds,  and  funding  TDYs  is  low 
priority . 

The  first  of  General  Randolph's  three  prime  goals  for 
Systems  Command  is  to  'Meet  the  user’s  needs.'  This  is  a 
serious  problem,  General  Randolph  wants  AFSC  people  to  meet 
with  the  user,  to  keep  close  to  the  user,  and  to  know  the 
user’s  needs  but  the  TDY  funds  are  not  there  to  do  it. 
Everything  done  at  HQ  AFSC  is  focused  on  the  goal  of  meeting 
the  user’s  needs. 

Detailed  requirements  generated  by  the  user.  Agree . 
This  is  definitely  necessary.  In  1987  the  requirements 
process  was  revised,  and  the  Requirements  Regulation  57-1  is 
the  result.  The  bottom  line  is  that  the  requirements 
process  is  now  in  the  hands  of  the  user.  In  the  past  the 
requirements  process  was  driven  by  HQ  AFSC. 

Under  the  new  process,  the  user  provides  a 
general /macro  statement  of  operational  need  (SON) .  HQ  AFSC 
takes  the  user’s  requirements,  in  the  form  of  the  SON,  and 
provides  the  options  and  alternatives  to  the  user  that 
fulfill  the  requirement.  The  cost  of  each  option  and 
alternative  is  provided. 

The  purpose  of  this  process  is  to  control  requirements 
creep.  By  preventing  the  PM  from  'feeding  the  user  . 
requirements',  the  contractor  is  no  longer  able  to  use  the 
PM  to  tell  the  user  about  'nice  to  have’  technology.  This 
greatly  reduces  the  number  of  user  requirements  that  are  not 
necessary  to  meet  the  user’s  SON. 

General  Randolph  established  the  Technology  and  Plans 
DCS  to  better  manage  the  technology  transition  and  ensure 
that  the  requirements  process  works  according  to  the  new 
regulation . 

The  other  primary  issue,  the  USAF  does  not  want 
detailed  requirements  from  the  user.  Requirements  are 
wanted  in  terms  of  need  instead  of  end  product.  This  will 
prevent  the  USAF  from  being  'locked  in'  to  any  one 
alternative.  Two  of  the  ten  criteria/questions  outlined  in 
AFSC  Acquisition  Strategy  Principles  to  evaluate  acquisition 
strategies  can  be  met  only  if  the  user’s  requirements  are 
from  a  macro  perspective  and  in  terms  of  need.  These  two 
questions  are:  1)  Does  the  strategy  represent  the  best 
value?,  and  2)  Have  alternatives  been  considered  throughout 
the  process? 

Problem  resolution  impacts  contractor  payment  schedule. 
TEAMWORK ; 


To  succeed  the  PM  must  preach  and  practice  teamwork. 
The  PM’s  team  must  go  beyond  SPO  walls.  The  team  must 
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include  the  AFPRO .  HQ  AFSC .  the  Pentagon,  and  the 
contractor . 

Very  important.  General  Randolph  has  established 
three  primary  goals  for  Systems  Command:  1)  meeting  users 
needs.  2)  maintaining  excellence  in  acquisition,  and  3) 
enhancing  technological  superiority.  General  Randolph 
proposes  to  implement  these  goals  through  better  teamwork 
and  improved  cooperation  between:  1)  users,  2)  HQ  AFSC,  3) 
SPO ,  and  4)  the  Pentagon  (SAE  and  DAE). 

One  of  the  current  problems  is  a  lack  of  effective 
teamwork  and  communication  at  the  staff  level.  The  proposed 
solution  is  to  focus  on  teamwork. 

Another  area  of  concern  is  keeping  AFSC  HQ  involved  in 
the  program  management  process.  This  is  seen  as  important 
to  ensure  that  the  necessary  resources  are  available  to 
support  Air  Force  prograots .  and  that  baselines  are 
executable.  General  Randolph  wants  to  dissolve  the 
separation  betvieen  HQ  and  the  rest  of  the  command.  He 
emphasizes  that  everyone  must  think  of  AFSC  as  one 
Integrated  unit  and  work  as  a  team.  One  of  the  primary 
vehicles  for  accomplishing  this  is  to  improve  communication. 

*A  good  PM  takes  advantage  of  teamwork.*  Holding  the 
PM  accountable  provides  the  incentive  for  the  PM  to  use  the 
elements  of  teamwork  in  program  management. 

A  second  teamwork  problem  arises  between  the  SPO  and 
AFPRO.  Physically  separated,  these  two  areas  often  do  not 
coordinate  or  communicate.  If  the  PM  is  on  the  ball,  and 
really  using  all  of  the  communication  channels  open  to 
him/her  then  he/she  will  use  the  AFPRO  because  of  the  wealth 
of  information  the  AFPRO  has  access  to. 

The  idea  of  contractor/government  team  effort  has 
always  been  promoted  at  AFSC.  On  the  other  hand,  the  Air 
Force  has  never  really  trusted  the  contractor.  This  lack  of 
trust  is  a  significant  problem  area.  It  is  hypothesized 
that  there  is  a  high  correlation  between  trust  and 
productivity,  therefore  teamwork  initiatives  (between  gov't 
and  contractors)  are  being  pursued. 

Looking  at  the  realities  of  the  situation,  every  time 
acquisition  policy-makers  are  making  headway  with  congress 
to  implement  these  teamwork  initiatives  another  skeleton 
falls  out  of  the  closet  and  Congress  adds  more  'legislative 
impediments.'  The  addition  of  'preemptive  oversight'  type 
legislation  prevents  the  PM  from  focusing  on  the  management 
principles  composing  these  'Menus  for  Success'.  Still, 
acquisition  policy  makers  are  trying  to  change  the  law  to 
facilitate  teamwork  between  the  government  and  contractors. 

All  constituents  involved  in  planning  and  scheduling. 

Yes 


Lateral  communication  channels  open.  Much  effort  is 
going  into  improving  this  area  in  AFSC. 
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Frequent  m««tin<a  used  to  resolve  differences.  Yes . 
The  idea  of  team  effort  betwreen  the  contractor  and  the 
government  has  always  been  promoted  at  AFSC .  That  is  why 
the  Air  Force  has  program  reviews.  On  the  other  hand,  the 
Air  Force  never  completely  trusts  the  contractor. 

Responsibilities  accepted,  not  assigned. 

One  face  to  users  and  management. 

All  functions  involved  in  problem  resolution.  This  is 
the  basic  premise  of  acquisition  streamlining.  By  placing 
responsibility  and  accountability  in  the  hands  of  the  PM, 
the  PM  has  the  incentive  to  establish  an  environment  that 
facilitates  this.  If  the  PM  does  not  work  with  the 
contractor  to  resolve  problems  then  the  chances  of  poor 
program  execution  increase  accordingly. 

PARTICIPATIVE  LEADERSHIP 


Site  representatives  involved  from  start  to  finish.  A 
good  idea.  Site  activation  teams  work  with  the  users. 

User  input  requested  in  all  areas  and  all  phases. 

Qood ,  agree . 

Site  input  actually  used.  Yes 

Users  kept  informed,  frequent  site  briefings  and  memos. 
Yea,  this  is  necessary. 

Problems  solved  at  the  lowest  level  possible.  Yes . 
Users  allowed  to  voice  disagreements  and  concerns. 


FOCUS 


Wo  bells,  whistles,  or  additional  fixes  were  added. 
Program  stability  is  very  important.  HQ  AFSC  has  done  a  lot 
of  work,  and  is  planning  to  do  a  lot  more.  They  are  pushing 
to  get  a  single  integrated  program  baseline  that  everyone 
signs  up  to.  The  baseline  is  the  document  that  stabilizes 
requirements.  It  takes  the  resources  available  through  the 
PPBS ,  and  ties  them  to  a  program  so  that  the  executive 
system  of  ‘oversight*  can  manage  the  program. 

The  acquisition  program  baseline  is  a  formal  agreement 
among  the  PD,  PEO ,  SAE.  Wo  baseline  changes  are  allowed 
unless  they  are  approved  by  the  SAE.  The  primary  purpose  is 
to  establish  commitment  and  accountability. 

Because  of  the  outside  influences  it  can  be  hard  for  a 
PM  to  manage  the  baseline.  Changes  caused  by  congressional 
funding  are  an  example  of  such  outside  influences  but,  this 
doesn't  negate  the  importance  of  the  baseline. 
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Innovation  reatricted  to  the  oriitinal  program  scope. 
Yes,  important. 

Studies  performed  by  support  contractors  limited. 

Agree.  The  use  of  support  contractors  in  the  Air  Force  is 
increasing  because  manning  is  decreasing.  Therd  is  nothing 
wrong  with  consultants,  someone  must  fill  the  void,  but  the 
PM  must  use  them  parsimoniously. 

COMTRACTOR  AMD  SYSTEM  MONITORIWQ 

The  major  focus  of  acquisition  policy  makers  in  this 
area  is  to  streamline  the  process.  There  are  several  HQ 
AFSC  initiatives  to  do  this.  Essentially,  the  gov’t  is 
trying  to  get  out  of  the  business  of  detailed  and  intensive 
contractor  monitoring. 

The  Model  Contractor  Program  is  a  new  initiative  to 
substitute  contract  remedies  and  good  warranties  for 
government  surveillance.  The  idea  is  to  allow  industry  to 
get  back  to  their  principal  role  of  producing  quality 
hardware  by  removing  some  of  the  oversight  requirements. 

The  expected  results  are  a  high  quality  product  and  an  iron¬ 
clad  warranty. 

The  Enterprise  Programs  are  another  effort  to 
streamline  the  acquisition  process  through  "regulatory 
relief."  Regulatory  relief  involves  waiving  whichever 
regulations  the  PM  can.  Certain  programs  are  selected  each 
year  based  on  the  stability  of  the  program.  Congress  then 
gives  the  program  multi-year  authorization,  not  to  be 
confused  with  multi-year  appropriation.  Every  year  the  list 
of  programs  being  managed  as  Enterprise  Programs  is  expected 
to  increase. 

For  the  enterprise  programs  streamlining  focused  on: 

1.  Reports  2.  Reviews/Briefings  3.  Budgeting 

4.  Testing  5.  Contracting  6.  Logistics 

7.  Data  8.  System  Architecture 

The  purpose  of  the  Flexible  Oversight  Initiative  is  to 
match  the  level  of  oversight  of  contractor  operations  to  the 
level  of  program  risk,  and  the  level  of  confidence  in 
contractor  performance.  Oversight  should  be  increased  if 
contractor  performance  deteriorates.  The  expected  results 
from  this  are  better  matching  of  oversight  requirements  and 
resources,  better  risk  oianagement,  and  reduced  contractor 
oversight  where  appropriate. 

The  primary  questions  behind  the  Contractor  Database 
Initiative  are:  1.  When  is  oversight  enough?,  and  2.  How 
much  should  you  trust  the  contractor?  In  Systems  Command 
they  are  reviewing  the  track  record  and  past  performance  of 
contractors,  and  building  a  database  with  the  information. 
The  purpose  of  the  database  is  to  assist  in  source  selection 
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decision*  by  providing  the  information  to  determine  who  the 
better  performers  are. 

The  Could  Cost  /  Total  Quality  Management  (TQM) 
Initiative  involves  the  joint  implementation  of  these  two 
concepts.  The  goal  is  to  improve  the  quality  and  lower  the 
coat  of  Air  Force  system  acquisitions.  These  concepts  are 
very  pervasive,  not  limited  to  acquisition. 

The  Could  Cost  concept  includes  the  realm  of  ‘should 
cost*.  The  focus  is  on  maximizing  efficiency  to  minimize 
cost.  The  basic  goal  is  to  identify  the  non-value 
requirements  levied  on  the  contractor  which  could  be 
realistically  reduced/eliminated . 

Quality  will  be  increased  through  acquisition 
incentives  that  encourage  contractor  performance  and  quality 
improvements.  Included  is  the  Air  Force  elimination  of 
fixed  quality  levels  in  specifications. 

Could  Cost  and  TQM  are  very  new  ideas.  Neither  has 
been  implemented,  both  are  still  in  the  concept  phase.  It 
is  not  known  how  enthusiastic  the  people  in  the  field  are 
going  to  be  about  the  concept.  Teamwork  with  industry  is 
essential  to  achieve  the  goals  of  this  initiative. 

Currently,  the  C-17  and  the  B-2  SPOs  are  actively  applying 
Could  Cost/TQM  to  improve  quality  and  cost  performance. 

Actual  demonstration  of  requirement  before  sign-off. 
Necessary. 

Technical  Performance  Measures  established  up  front. 
Yes,  the  new  requirements  process,  TEMP,  outlines  this  as 

necessary. 

RMA  established  up  front,  and  achievable.  Yes ,  the  new 
requirements  process,  TEMP,  outlines  this  as  necessary. 

All  problems  categorized  and  visible  in  INFO  database. 
Maybe,  really  not  too  familiar  with  this  area. 

Numerous  indicators  of  program  progress  used.  Maybe . 
TESTING  AND  TRAINING 


Testing  as  early  in  development  as  possible.  Good 
point.  HQ  AFSC  is  currently  putting  a  lot  of  emphasis  on 
getting  an  early  start  on  testing.  It  is  important,  and 
should  be  done.  A  new  requirements  process  called  TEMP  has 
been  instituted  to  get  planning  for  testing  up  front.  TEMP 
is  a  requirements  correlation  matrix  that  shows  testing 
requirements,  specifications,  thresholds,  and  goals.  It 
helps  to  'nail  down*  the  testing  process. 
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A  lot  of  factors  play  against  early  testing,  some  of 
the  major  factors  are  outlined  below: 

1.  For  the  PM,  the  more  test  requirements  levied 
on  him,  the  more  chances  that  the  program  will  be 
slowed  down. 

2.  Test  requirement  funding,  the  more  testing  that 
is  done,  the  greater  the  cost. 

3.  What  if  problems  are  found.  Often  the  team  is 
afraid  of  finding  problems.  They  are  afraid  that  they 
may  not  like  the  answer,  it  may  jeopardize  the  program. 
This  is  the  political  aspect. 

Both  structured  and  unstructured  testing  conducted. 

‘The  Air  Force  currently  sticks  to  structured  testing  as  far 
as  I  know. *  It  is  more  important  to  have  a  good  structure 
than  performing  unstructured  testing.  A  good  structure 
allows  planning  for  testing,  and  early  testing.  The  real 
problem  is  that  the  Air  Force  does  not  structure  tests  well. 

User  involved  in  planning  and  performing  test 
scenarios .  Yes,  this  is  important. 

Site  personnel  trained  before  system  delivery. 
Important.  Most  PMs  talk  about  it,  but  it  is  not  done  too 
much.  One  of  the  primary  limiting  factors  is  funding 
constraints. 

After  delivery,  detailed  site  specific  testing 
conducted .  Yes,  this  is  necessary.  Operational  test  and 
evaluation  is  currently  performed  in  Air  Force  programs. 

CONCLUSION 


Everything  here  has  been  thought  of  long  before.  What 
has  the  potential  to  be  different  is  the  method  of 
implementation  and  the  people.  There  are  things  that  the 
Air  Force  has  done  along  these  lines,  and  those  things  are 
good,  but  there  is  still  a  long  way  to  go. 

Some,  but  not  all,  of  the  current  HQ  AFSC  initiatives 

are : 

1.  Acquisition  Reporting 
■2.  Baseline  Consolidation 

3.  Could  Cost  and  Total  Quality  Management  (TQM) 

4.  Acquisition  Regulation  Streamlining 

5.  Flexible  Oversight 

6.  Model  Contractor  Program 

7.  Acquisition  Information  Structure 


199 


NAME:  Mr.  Ira  Kemp 


TITLE:  Associate  Director 

Directorate  of  Contracting  and  Manufacturing  Policy 

SAF/AQC 

The  Pentagon 

Washington,  D.C.  20330 

DATE:  29  June  1988 

TIME:  1000 

PLACE:  The  Pentagon 


CONTRACT 


Award  Fee  and  Incentive  Fee  used  jointly  (CPIF/AF) . 
Nothing  new,  used  frequently.  This  is  used  as  an 
opportunity  to  get  up  close  and  personal  with  the 
contractor.  Provides  some  incentive,  but  not  a  great  deal. 

Waterfall  implementation  plan.  Of  course  this  is 
necessary.  It  is  always  effective  to  'build  on  the  learning 
curve'  and  have  the  core  team  move  from  site  to  site. 

Realistic  schedule  providing  some  slack.  This  is 
smart,  there  is  certainly  nothing  wrong  with  this.  I  am  an 
advocate  of  realistic  schedules.  The  problem  is  that  too 
many  programs  have  unrealistic  IOC  dates  without  full  regard 
for  budget/cost  and  technical  considerations. 

Funds  budgeted  to  support  participative  requirements. 
Important.  Either  the  user  or  the  PM  should  be  considering 
these  requirements  in  their  budget  estimates. 

Detailed  requirements  generated  by  the  user. 
Conceptually  the  user  generates  the  SON,  and  the  procuring 
agency  then  provides  the  user  with  the  alternatives  that 
meet  the  SON.  A  decision  would  then  be  made  based  on  the 
best  alternative  solution.  In  reality  it  will  be 
predetermined  how  to  best  accomplish  the  Job.  Also,  if  the 
number  of  alternatives  to  meet  a  user's  need  are  fairly 
limited,  then  the  SON  pretty  much  defines  the  requirements. 
But  again,  according  to  current  procedures,  the  user  only 
generates  the  SON,  not  the  specific  requirement. 

Problem  resolution  impacts  contractor  payment  schedule. 
This  is  a  good  incentive.  A  good  management  technique,  and 
a  good  attempt  to  organize  and  manage  problems. 

TEAMWORK 

All  constituents  involved  in  planning  and  scheduling. 

A  nice  idea,  but  be  sure  that  someone  is  in  charge. 
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Lateral  conununication  channels  open.  All  channels  must 
be  open  for  the  program  to  function  properly.  People  must 
feel  free  to  use  informal  channels,  but  management  must  be 
kept  informed. 

Frequent  meetings  used  to  resolve  differences.  Yes , 
but  with  a  defined  agenda  known  by  the  parties  involved. 

Responsibilities  accepted,  not  assigned.  People  must 
know  what  they're  responsible  for.  some  will  do  less  and 
some  will  do  more.  Assigned  versus  accepted  depends  on  the 
quality  of  the  individual ,  practicality  in  staffing,  and 
trust  in  people. 

One  face  to  users  and  management.  This  generally 
works.  There  is  always  the  potential  that  the  contractor  is 
impacted  by  someone  influential  who  gets  the  contractor  off 
of  the  track.  It  is  up  to  the  PM  to  keep  the  contractor  in 
line,  and  to  get  the  contractor  to  understand  the  specific 
sources  that  guidance  and  changes  can  come  from.' 

All  functions  involved  in  problem  resolution.  Maybe 
this  is  OK,  maybe  it  is  not.  The  PM  gets  paid  for  weighing 
alternatives.  Kicking  the  contractor  is  not  always  the 
thing  to  do.  Every  so  often  the  PM  must  go  the  extra  mile 
and  help  the  contractor  to  get  over  the  hump. 

PARTICIPATIVE  LEADERSHIP 


Site  representatives  involved  from  start  to  finish. 

Yes.  The  award  fee  facilitates  this.  It  gives  the  user  the 
ability  to  provide  input  for  the  evaluation  of  the 
contractor . 

User  input  requested  in  all  areas  and  all  phases.  Yes . 
User  input  is  very  important.  It  is  a  good  source  of 
information  for  the  PM. 

Site  input  actually  used.  Yes . 

Users  kept  informed,  frequent  site  briefings  and  memos. 
Yes,  definitely. 

Problems  solved  at  the  lowest  level  possible.  Yes .  I 
certainly  agree.  If  this  was  not  done,  then  the  higher 
levels  of  management  would  not  survive. 

Users  allowed  to  voice  disagreements  and  concerns. 
Certainly . 
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FOCUS 


Mo  bella,  whiatlea,  or  additional  fixes  were  added.  I 
can  accept  this.  This  used  to  be  a  much  bigger  problem, 
baselining  holds  this  under  control  now.  It  is  important  to 
evaluate  each  item:  Do  you  need  it  because  you  would  like  to 
have  it?  or  Do  you  need  it  to  make  the  program  work? 

Innovation  restricted  to  the  original  problem  scope. 

Studies  performed  by  support  contractors  limited. 
CONTRACTOR  AND  SYSTEM  MONITORING 

It  depends.  ‘Micromanagement*  is  sometimes  good,  and 
sometimes  bad.  The  convolutions  and  conflicts  are 
incredible.  There  are  no  truisms  in  this  area. 

The  current  initiatives  to  streamline  may  or  may  not 
impact  this  area.  The  idea  is  to  reduce  the  degree  of 
contractor  monitoring,  but  in  many  cases  the  number  of  steps 
are  reduced  but  the  amount  of  detail  within  each  step  is 
increased . 

Actual  demonstration  of  requirement  before  sign-off. 

Technical  Performance  Measures  established  up  front. 

RMA  established  up  front,  and  achievable. 

All  problems  categorized  and  visible  in  INFO  database. 

Numerous  indicators  of  program  progress  used. 

TESTING  AND  TRAINING 


Testing  as  early  in  development  as  possible.  Yes .  You 
have  got  to  know  whether  the  criteria  are  met  ASAP.  That 
way,  if  changes  are  needed  they  can  be  cranked  in.  It  is 
important  to  remain  flexible  I  Testing  will  cause  you  to 
make  changes  and  tradeoffs  within  the  technical  and  budget 
baseline . 

Both  structured  and  unstructured  testing  conducted. 

Yes.  The  Air  Force  tries  to  approach  unstructured  testing 
in  a  more  logical  method.  You  can  'crash'  a  computer,  but 
you  can't  crash  an  airplane.  With  aircraft  the  envelope 
must  be  expanded  gradually.  The  computer  is  a  safer 
environment . 

User  involved  in  planning  and  performing  test 
scenarios .  Absolutely. 
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Site  peraonnel  trained  before  system  delivery.  This  is 
necessary,  and  always  intended.  Without  this  the  system  is 
useless.  The  most  crucial  part  of  any  system!  For  example, 
an  F-15  is  relatively  easy  to  deliver,  but  finding  pilots  to 
fly  it  and  maintenance  people  to  repair  it  is  very  hard  if 
they  have  not  been  trained. 

After  delivery,  detailed  site  specific  testing 
conducted .  Yes . 


COMCLUSION 


What  works  for  one  PM  may  not  work  for  another.  It 
depends  on  the  PM  and  the  organization.  Still,  there  has  to 
be  structure  and  guidance.  There  must  be  parameters  defined 
for  decisions.  As  long  as  they  have  that,  an  organization 
can  move  forward.  But,  policy-makers  must  be  careful  not  to 
overload  the  organization,  or  it  will  stop  dead. 

Program  managers  can  not  merely  rely  on  common  sense  to 
manage  a  program  because  too  many  people  believe  in  rules 
and  legislation.  If  you  do  what  makes  sense,  it  invariably 
won't  make  sense  to  auditors,  the  IG,  the  GAO  etc.  Before 
you  know  it,  all  of  these  organizations  will  be  screaming 
■  fraud. ■ 
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NAME:  Colonel  (Retired)  James  Lindenfelser 


TITLE:  Manager  of  the  Defense  Acquisition 
Management  Department 
Systems  Management  Group 
The  Analytic  Sciences  Corporation 
1700  North  Moore 
Suite  1800 

Arlington.  VA  22209 

DATE:  8  July  88 

Tllffil:  0900 

PLACE:  The  Analytic  Sciences  Corporation  (TASC) 


CONTRACT 

Award  Fee  and  Incentive  Fee  used  jointly  (CPIF/AF) . 
Agree,  both  are  good. 

Waterfall  implementation  plan. 

Realistic  schedule  providin<  some  slack. 

Funds  budgeted  to  support  participative  requirements. 
This  is  not  really  applicable.  It  is  not  important  whether 
the  user’s  funds  or  the  SPO’s  funds  are  used. 

Detailed  requirements  generated  by  the  user.  No,  not 
detailed.  The  user  should  generate  the  need,  not  detailed 
requirements.  Need  statements  should  not  contain  detailed 
system  descriptions.  This  allows  competition  between 
concepts,  and  promotes  nontraditional  and  innovative 
solutions.  Too  often  DOD  need  statements  do  contain  weapon 
system  solutions,  which  is  not  optimal. 

The  user  and  the  program  management  team  should  work 
together  to  collectively  make  the  tradeoffs  between 
alternatives  to  determine  how  to  best  satisfy  the  need. 
Further,  the  user  must  be  involved  in  determining  if  the 
specifications  selected  meet  their  need. 

_ Problem  resolution  impacts  contractor  payment  schedule. 

A  successful  program  has  to  have  a  cooperative  contractor. 
Cooperation  can  be  achieved  through  this  and  similar 
contract  elements. 

TEAMWORK 


Teamwork  is  Important,  needed,  and  the  only  way  to  do 
business.  All  successful  programs  have  teamwork.  However, 
teamwork  does  not  guarantee  a  successful  program.  Some 
unsuccessful  programs  have  good  teamwork  too. 
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In  the  DOD  the  definition  of  teamwork  is  very  broad. 

It  includes  the  efforts  of  the  SPO,  training  command,  Air 
Staff,  the  Secretariat,  the  Office  of  the  Secretary  of 
Defense,  Congress,  the  PM  and  the  user.  Teamwork  can  be 
used  in  DOD  to  ensure  program  environment  stability.  A 
disruptive  environment  kills  both  the  program  and  the  PM. 

In  addition  to  teamwork,  a  cooperative  contractor,  a 
stable  environment,  etc,  it  is  also  important  for  the  PM  to 
understand  the  contractor,  and  the  contractor’s  systems  and 
engineering  functions. 

The  concept  of  teamwork  within  the  SPO  should  also  be 
addressed.  The  PM  must  trust  his/her  people,  and  hold  them 
accountable  for  the  responsibilities  assigned. 

The  teamwork  between  the  government  and  the  contractor 
is  directly  affected  by  external  factors.  Every  time  DOD 
gets  bad  publicity,  the  relationship  between  the  service  and 
the  contractor  gets  strained.  Environment  is  very 
important.  Luck  and  timing  are  key  players  in  this  area. 
Poor  publicity  in  one  area  of  DOD  adversely  impacts  other 
areas . 

All  constituents  involved  in  planning  and  scheduling. 

Lateral  communication  channels  open. 

Frequent  meetings  used  to  resolve  differences. 

Responsibi 1 ities  accepted,  not  assigned. 

One  face  to  users  and  management. 

All  functions  involved  in  problem  resolution. 
PARTICIPATIVE  LEADERSHIP 

A  PM  has  got  to  have  the  user  on  his/her  team. 

Site  representatives  involved  from  start  to  finish. 

Very  good  and  very  important.  One  of  the  reasons  that  the 
Air  Force  has  moved  towards  the  baseline  concept  is  to 
increase  user  involvement.  The  intent  is  to  match  programs 
content,  dollars  and  schedule  to  promote  understanding  and 
stability . 

User  input  requested  in  all  areas  and  all  phases. 

Site  input  actually  used. 

Users  kept  informed,  frequent  site  briefings  and  memos. 

Problems  solved  at  the  lowest  level  possible. 

Users  allowed  to  voice  disagreements  and  concerns. 
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FOCUS 


All  of  those  elements  ere  important. 

Wo  bells,  whistles,  or  additional  fixes  were  ad'-'ed. 


Innovation  restricted  to  the  original  problem  scope. 
Studies  performed  by  support  contractors  limited. 


CONTRACTOR  AND  SYSTEM  MOWITORIWG 


The  degree  of  program  oversight  should  be  the  minimum 
necessary  to  ensure  that  the  program  objectives  are  met. 
Program  objectives  include  schedule,  budget  and  field 
performance.  The  PM  must  work  in  unison  with  the  contractor 
to  determine  the  degree  of  oversight  necessary. 

Actual  demonstration  of  requirement  before  sign-off. 

Technical  Performance  Measures  established  up  front. 

RMA  established  up  front,  and  achievable. 

All  problems  categorized  and  visible  in  INFO  database. 

A  good  management  technique.  All  good  PMs  should  have  a 
system  to  do  this. 

Numerous  indicators  of  program  progress  used. 

TESTING  AND  TRAINING 


Philosophically  these  elements  are  in  league  with  DOD 
objectives.  In  actuality,  DOD  is  having  a  hard  time  meeting 
these  elements.  Testing  especially  has  been  under  scrutiny. 
DOD  has  a  testing  agency  with  an  independent  tester  in  each 
service . 

Testing  as  early  in  development  as  possible. 

Definitely  important.  Testing  is  an  integral  part  of 
process  and  system  monitoring.  This  element  applies  to  both 
development  and  operational  testing.  DOD  is  trying  to  head 
in  this  direction.  Operational  testing  conducted  early  in 
development  is  difficult,  but  it  can  be  done  thru 
simulations  and  modeling. 

Both  structured  and  unstructured  testing  conducted.  I 
like  this  element. 

User  involved  in  planning  and  performing  test 
scenarios .  Absolutely  imperative. 
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Site  personnel  trained  before  system  delivery.  This  is 
very  important.  This  element  can  not  be  an  afterthought. 
Training  must  be  an  integral  part  of  the  program. 

After  delivery,  detailed  site  specific  testing 
conducted . 

COMCLUSION 

All  of  these  things  are  good  things,  but  both  funding 
stability  and  requirements  stability  must  be  there  first. 

For  a  program  to  be  successful  the  PM  has  got  to  have  the 
user  on  the  team,  and  there  has  to  be  baseline,  budget,  and 
environment  stability.  If  any  one  of  these  four  elements  is 
missing,  the  program  is  likely  to  fail. 

The  primary  problem  with  this  list  of  program 
management  elements  identified  by  the  Host  program 
management  team  is  that  they  can  all  be  found  in 
unsuccessful  programs  too.  These  elements  don’t  guarantee 
success.  A  successful  program  has  all  of  these  menu  items 
plus  a  few  additional  items,  especially  stability.  It  is 
also  important  to  remember  that  in  DOD  some  things  that  are 
good  management  precepts  don’t  work  as  well  as  expected 
because  of  DOD’s  environment. 

A  good  program  manager  doesn’t  necessarily  make  a 
program  successful,  neither  does  good  management.  Having  a 
few  unsuccessful  programs  isn’t  bad,  and  doesn’t  always 
Indicate  poor  management.  Programs  that  are  not  achieving 
their  primary  objectives  should  be  cancelled,  and  the  PM 
should  be  rewarded  if  he/she  had  managed  the  program  well. 

Of  course  this  is  not  what  happens  in  DOD. 

These  areas  should  be  considered  for  inclusion  in  the 
program  management  'Menus  for  Success*  developed  by  this 
research : 

1.  Emphasize  stability  more. 

2.  Quality  personnel  are  a  key  element  to  program 
success.  'Black'  programs  work  better  because  they  get 
better  people.  Address  the  people  issue  more  in  the 
elements  from  Host. 


3.  The  PM  must  understand  the  POM,  resource 
allocation,  process.  Without  this  understanding  the  PM  is 
lost.  It  is  within  this  process  that  the  PM  can  work  to 
maintain  stability.  Resource  allocation  is  the  'name  of  the 
game.'  The  PM  has  little  control  over  output,  but  input  can 
be  provided. 

4.  The  need  to  track  system  performance  and 
program  schedule  must  be  added  to  the  Contractor  and  System 
Monitoring  elements. 


5.  In  addition  to  the  other  Teamwork  elements,  the 
PM  must  understand  the  contractor,  their  monitoring  system, 
and  what  is  being  done  in  engineering.  Any  successful 
program  requires  this.  FAA  most  likely  did  this  up  front 
too . 

6.  A  cooperative  contractor  is  vital.  Contract 
elements  can  be  used  to  motivate  a  contractor  to  cooperate. 

7.  Luck,  in  terms  of  Judgement  and  external 
factors,  plays  a  major  role  in  program  success.  Competency 
and  luck  go  hand  in  hand.  There  is  a  mystical  quality  of 
this  entire  thing.  I  would  rather  be  lucky  than  good.  PMs 
can  contribute  to  their  own  good  luck.  The  right  team, 
right  PM  and  right  congressman  speaking  up  at  the  right  time 
all  contribute  to  luck.  The  right  program  manager  is  also 
an  issue  because  most  DOD  PMs  are  relatively  untested  as  PMs 
before  they  get  their  first  program. 

The  uncontrollable  items  are  an  important  factor  in  the 
success  of  the  program.  For  example,  you  are  a  PM,  doing  a 
good  job  managing  your  program,  when  some  fraudulent 
practices  are  discovered  in  a  program  completely  unrelated 
to  yours.  All  of  a  sudden  your  program  or  primary 
contractor  is  affected  because  one  of  their  other  branches 
is  under  investigation. 

The  very  nature  of  judgement  implies  more  than  one 
alternative  and  multiple  outcomes.  Picking  the  alternative 
with  the  best  outcome  requires  as  much  knowledge  as  possible 
supported  by  an  intuitive  sense.  This  is  an  integral  part 
of  being  a  good  PM.  The  PM  must  know  how  to  work  the 
system!  For  example,  hitting  for  more  funds  at  the  right 
time,  or  having  a  problem  and  knowing  when  to  strike. 


NAME:  Mr.  Daniel  S.  Bak 

TITLE:  Deputy  Asaistant  Secretary  of  the  Air  Force 
(Acquisition  Management  and  Policy) 
SAF/AQ 

The  Pentagon 
Washington,  D.C.  20330 

DATE:  1  July  88 

TIME:  1500 

PLACE:  The  Pentagon 


CONTRACT 


Award  Fee  and  Incentive  Fee  used  jointly  (CPIF/AF) . 
Should  be  based  on  the  degree  of  risk  involved.  The  share 
ratio  between  IF  and  AF  can  be  varied.  This  combination 
gives  the  maximum  latitude  in  management  and  flexibility. 

The  Contracting  Officer  and  Program  Manager  should  be 
innovative,  but  they  should  not  abandon  the  basic  contract 
structure  and  elements. 

Waterfall  implementation  plan.  Yes,  this  is  necessary 
and  it  is  currently  used  by  DOD. 

Realistic  schedule  providing  some  slack.  Yes,  this  is 
necessary.  Unfortunately  the  Program  Manager  is  often 
pushed  into  an  unrealistic  schedule. 

The  baselining  process  is  a  new  innovation  which  may  be 
able  to  force  a  more  realistic  schedule  from  the  user.  The 
chain  of  command  has  been  shortened  for  the  baseline 
approval.  The  PM  is  now  held  accountable  for  meeting  the 
baseline.  Before  the  PM  signs  the  baseline,  the  user  has  to 
sign  off  on  the  baseline. 

Under  the  old  baselining  system,  baselines  were 
cumbersome  and  hard  to  get  into  place.  Under  the  new 
baselining  process  there  is  a  clear  and  direct  line  between 
the  PM  and  the  Service  Acquisition  Executive  (SAE) .  This 
process  will  force  the  PM  and  the  user  to  jointly  establish 
a  firm  and  realistic  schedule. 

Funds  budgeted  to  support  participative  requirements. 
This  might  be  nice.  Currently  it  is  up  to  the  user  to 
budget  for  and  fund  their  own  TDYs  in  support  of  the 
program. 

Detailed  requirements  generated  by  the  user. 

Problem  resolution  impacts  contractor  payment  schedule. 
Not  really  a  parallel  from  DOT  to  DOD. 
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TEAMWORK 


Teamwork  is  necessary.  To  the  extent  that  DOD  can,  it 
does  use  teamwork.  Teamwork  is  also  emphasized  within  the 
SPO.  The  degree  of  teamwork  necessary  and  appropriate  is 
dependent  on  the  type  of  contract. 

All  constituents  involved  in  planning  and  scheduling. 

Lateral  communication  channels  open. 

Frequent  meetings  used  to  resolve  differences. 

Responsibilities  accepted,  not  assigned. 

One  face  to  users  and  management. 

All  functions  involved  in  problem  resolution.  There 
are  times  when  this  would  not  be  appropriate.  It  depends  on 
how  much  responsibility  the  government  wants  to  take  for  the 
solution.  Therefore,  the  PM  must  proceed  cautiously.  It 
also  goes  back  to  what  the  contractor  is  being  paid  to  do. 

PARTICIPATIVE  LEADERSHIP 

Agree,  this  is  necessary.  DOD  is  currently  encouraging 
even  more  user  participation. 

Site  representatives  involved  from  start  to  finish.  On 
major  programs,  a  user  representative  is  co-located  with  the 
SPO. 

User  input  requested  in  all  areas  and  all  phases. 

Site  input  actually  used. 

Users  kept  informed,  frequent  site  briefings  and  memos. 
Problems  solved  at  the  lowest  level  possible. 

Users  allowed  to  voice  disagreements  and  concerns. 

FOCUS 


One  of  the  reasons  that  the  new  baseline  process 
requires  that  the  baseline  be  firmed  at  a  higher  level  than 
before  is  to  inhibit  the  user  from  changing  his/her 
requirements . 

The  new  requirements  development  procedures  will  also 
help  to  stabilize  requirements. 

No  bells,  whistles,  or  additional  fixes  were  added. 
Absolutely,  no  question. 
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Innovation  reatricted  to  the  original  problem  scope. 
Absolutely,  no  question. 

Studies  performed  by  support  contractors  limited.  This 
definitely  makes  sense. 

COMTRACTOR  AND  SYSTEM  MOMITORING 

The  degree  required  depends  on  the  program,  but  it  is 
essential  to  every  program  irregardless  of  degree. 

Actual  demonstration  of  requirement  before  sign-off. 

Technical  Performance  Measures  established  up  front. 

RMA  established  up  front,  and  achievable. 

All  problems  categorized  and  visible  in  INFO  database. 
Good  idea. 

Wumerous  indicators  of  program  progress  used. 

TESTIMQ  AWD  TRAIWIMG 

Testing  as  early  in  development  as  possible.  Good , 
needed . 

Both  structured  and  unstructured  testing  conducted. 
Good,  needed. 

User  involved  in  planning  and  performing  test 
scenarios .  Good ,  needed . 

Site  personnel  trained  before  system  delivery.  Good , 
needed . 

After  delivery,  detailed  site  specific  testing 
conducted .  Good,  needed. 

CONCLUSIOM 


This  program  had  such  incredible  flexibility,  no  wonder 
it  was  successful. 

DOO  currently  does  something  in  every  category  here. 
What  is  done  depends  on  which  acquisition  phase  the  program 
is  in.  The  most  important  key  to  a  successful  program  is  a 
realistic  schedule.  If  you  have  this,  you  can  make  about 
any  program  successful.  Next  in  importance  is  cost 
f lexibi 1 ity . 
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NAME:  Colonel  Ralph  Tourino 


TITLE:  Inspector  General 
AFSC/IG 

Andrews  AFB.  D.C.  20334 

DATE:  30  June  1986 

TIME:  1000 

PLACE:  HQ  AFSC ,  Andrews 


CONTRACT 

If  the  Air  Force  had  all  of  these  contract  elements 
that  Host  had.  then  Air  Force  acquisitions  would  be  more 
successful.  The  reason  that  they  wouldn’t  have  some  of 
these  elements  is  external,  out  of  the  PM  and  CO’s  control. 

Award  Fee  and  Incentive  Fee  used  jointly  (CPIF/AF) . 

A jree .  All  contracts  should  have  Award  Fees.  They  are  a 
key  management  tool  to  influence  a  contractor’s  manning, 
staffing  and  problem  resolution.  A  big  proponent  of  Award 
Fee  type  contracts.  Incentive  Fees  not  seen  as  quite  so 
beneficial  since  the  people  who  take  action  to  meet  the 
incentive  won’t  be  there  to  reap  the  benefits,  but  with 
Award  Fees  the  person  who  takes  the  action  will  be  there  to 
reap  the  benefits. 

Award  Fees  require  discipline  from  both  the  SPO  and  the 
contractor.  It  is  a  neat  tool,  it  can  give  the  Program 
Director  latitude  since  it  is  qualitative.  Qualitative  is 
the  Award  Fee’s  strength.  As  a  caution,  proper  use  of  Award 
Fees  requires  trust  on  both  sides.  The  contractor  must  view 
it  as  fundamentally  fair. 

Waterfall  implementation  plan.  OK 

Realistic  schedule  providing  some  slack.  This  is 
essential.  It  is  essential  to  bringing  a  contract  in.  A 
significant  problem  with  major  system  acquisitions  is  an 
unrealistic  schedule.  An  unrealistic  schedule  is  the  result 
of  optimism  and/or  a  need  or  threat  identified  by  the  user 
or  Pentagon. 

Most  programs  in  trouble  have  unrealistic  schedules. 

For  example,  the  engineering  has  not  been  completed  before 
testing  was  begun. 

Funds  budgeted  to  support  participative  requirements. 
This  is  needed.  It  would  be  nice  and  very  good,  but  AFSC 
does  not  usually  get  the  budget  to  do  this.  What  is 
fundamental  to  successful  major  systems  acquistion  is  budget 
stabi 1 i ty . 
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Detailed  reguirementa  ftenerated  by  the  user.  This  is 
essential.  User  Involvement  in  requirements  definition  Is 
vital.  The  level  of  detail  required  depends  on  the  maturity 
of  the  program,  and  on  what  is  being  procured.  In  the  Air 
Force  the  user  generates  the  need,  and  as  the  engineering 
evolves  so  does  the  detail  of  the  user’s  knowledge  which 
leads  to  more  detailed  requirements.  If  there  are  no  broad 
alternatives  to  meeting  the  user’s  need,  as  in  a  rehosting, 
then  the  user  would  be  able  to  write  very  detailed 
requirements. 

Stability  of  requirements  is  very  important.  The  user 
presents  their  Statement  of  Heed  (SOH)  at  the  very  start  of 
the  acquisition  process.  The  AFSC  then  provides 
alternatives  that  meet  the  SON.  The  user  selects  the 
alternatives  and,  engineering  development  begins.  As  the 
engineering  matures,  the  user  refines  his  requirements. 

This  is  an  evolutionary  process  which  leads  eventually  to  a 
field  weapon  system. 

Problem  resolution  impacts  contractor  payment  schedule. 


TEAMWORK 


Teamwork  is  the  proper  term  for  how  the  government  and 
contractor  should  interact.  In  today’s  environment  the  Air 
Force  is  being  criticized  for  being  too  close  to 
contractor^.  A  few  years  ago  it  was  for  being  too 
adversarial. 

Actually  teamwork  is  a  delicate  balance,  a  large  grey 
area.  It  is  important  to  cooperate,  but  also  important  to 
maintain  an  'arms  length'  relationship.  It  is  very 
counterproductive  when  a  Program  Director  forgets  this. 

All  constituents  involved  in  planning  and  scheduling. 
Very  good.  Probably  a  good  lesson  learned.  Applicable  to 
the  Air  Force,  but  not  really  used.  Basically  getting  a 
baseline  to  manage  the  program  against.  The  way  that  this 
was  handled  with  Host  is  very  interesting,  ie.  having 
intensive  discussions  with  all  participants  immediately 
after  contract  award.  This  is  something  that  the  DOD 
usually  does  not  do  well.  After  the  contract  is  awarded  the 
pressure  to  dig  in  and  understand  what  the  SOW  actually  says 
is  off.  Also,  contractors  do  not  man-up  early  enough.  The 
main  point  is  the  necessity  of  getting  a  'meeting  of  the 
minds .  ' 

Lateral  communication  channels  open.  Award  Fee  forces 
communication,  and  can  also  facilitate  teamwork.  Essential. 

Frequent  meetings  used  to  resolve  differences. 

Recommend  rewording  this  element  to,  meetings  frequent, 
people  prepared  and  action  items  tracked.  This  is  good  only 
if  people  are  prepared  for  the  meeting,  and  if  action  items 
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are  tracked.  Frequent  meetings  are  not  a  prescription  for 
success  themselves.  More  important,  both  sides  prepared  to 
resolve  problems  at  the  meeting.  Meetings  are  important  to 
keep  the  channels  of  communication  open,  but  everyone  needs 
to  know  why  they  are  there,  and  the  action  items  need  to  be 
tracked . 

Responsibilities  accepted,  not  assigned. 

One  face  to  users  and  management. 

All  functions  involved  in  problem  resolution.  Yes . 
Adversity  sometimes  builds  teamwork. 

PARTICIPATIVE  LEADERSHIP 


The  Air  Force  climate  is  changing  from  minimal  to 
maximum  user  participation.  The  user  is  now  involved  in 
evaluating  changas .  and  has  the  responsibility  to  make  the 
decisions.  Sosie  major  SPOs  have  full  time  co-located  users. 
This  facilitates  communication,  and  reduces  the  chance  of 
misunderstanding.  I  don’t  know  if  we  do  it  enough,  it 
depends  on  the  size  of  the  program. 

User  involvement  is  very  important.  Continuous  dialog 
with  the  user  on  requirements  is  also  very  important.  The 
user  has  an  operational  perspective  while  the  SPO  has  a 
developer’s  perspective.  The  user  can  help  identify  what 
the  SPO  sometimes  downplays,  spares,  documentation,  and  help 
make  sure  that  the  system  is  supportable. 

It  is  also  effective  to  use  the  same  techniques 
outlined  in  this  category  to  Involve  the  SPO  with  the 
contractor.  The  Air  Force  has  found  it  effective  to  send 
SPO  people  out  to  the  contractor’s  plant  to  stay  for  one  or 
two  months.  A  new  individual  is  rotated  in  as  the  old  is 
rotated  out.  The  SPO  and  AFPRO  need  to  work  as  a  team  and 
this  plan  will  not  supplant  that.  The  AFPRO  is  a  monitoring 
function,  and  the  SPO  individual  helps  to  integrate  the 
engineering  functions.  Also,  the  AFPRO  does  not  work  for  the 
PM,  but  the  SPO  people  do.  Therefore  the  PM  can  direct  what 
they  monitor  and  the  PM  is  more  assured  of  ‘lead  time'  for 
engineering  problems.  This  could  be  used  very  effectively 
as  preventative  medicine,  for  the  daily  running  of  a  large 
program.  From  experience,  this  is  very  hard  to  do,  but  very 
ef  f active . 

Site  representatives  involved  from  start  to  finish. 

Yes 


User  input  requested  in  all  areas  and  all  phases.  Yes 

Site  input  actually  used.  Very  important.  Otherwise 
the  user  will  see  right  through  you. 
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Uaera  allowed  to  voice  diaa^reementa  and  concerns. 


FOCUS 


It  ia  important  to  be  tough  on  additional  acope  and 
atudiea.  Thia  ia  ‘right  on.*  You  need  to  keep  the 
contractor  and  SPO  both  focuaed  on  the  heart  of  the  problem 

No  bella.  whiatlea,  or  additional  fixea  were  added. 

Innovation  reatricted  to  the  original  problem  scope. 

Studies  performed  by  support  contractors  limited. 
Interesting  catch,  never  seen  written  down  before.  The 
first  new  thing.  Agree,  studies  can  dilute  an  effort 
significantly.  It  is  important  to  be  tough  on  these. 

CONTRACTOR  AND  SYSTEM  MONITORING 

It  is  important  not  to  tell  the  contractor  how  to  do 
the  work.  Don't  impose  our  ‘neat  ideas*  on  the  contractor 
for  engineering  fixes.  It  is  the  SPO’s  job  to  monitor 
progress,  not  to  provide  solutions.  As  long  as  the  SPO 
fundamentally  believes  what  the  contractor  is  doing  will 
work  it  is  OK  to  present  alternatives,  but  once  presented 
leave  the  contractor  alone  with  it.  But,  if  the  SPO  knows 
that  the  contractor's  method  will  fail  then  the  PM  must  tell 
them.  If  a  contractor's  method  is  very  risky  then  the  PM 
should  tell  them,  and  have  proof  to  show  them. 

The  program  manager's  job  is  to  keep  the  program  in  a 
‘risk  basket*  that  is  acceptable.  The  SPO  should  do  their 
own  engineering  analysis  in  the  areas  that  are  risky. 

Actual  demonstration  of  requirement  before  sign-off. 

Technical  Performance  Measures  established  up  front: 
Agree.  This  is  necessary  and  it  should  be  done  up  front  and 
jointly.  If  these  are  not  in  place  at  the  beginning,  then 
‘the  cows  are  already  out  of  the  barn*  by  the  time  you  get 
the  data.  In  addition  to  the  data  as  to  why  we  are  in 
trouble,  we  also  need  lead  time  notice  of  problems.  These 
should  evolve  as  the  SPO  ‘gets  smarter.* 

Software  design  indicators  are  a  ‘black  art.'  We  are 
just  now  getting  a  handle  on  these,  yet  they  are  fundamental 
to  bringing  in  an  effort  like  Host. 

RMA  established  up  front,  and  achievable. 


All  problems  categorized  and  visible  in  INFO  database. 


Nxmerous  indicators  of  pro><ram  progress  used.  Agree , 
it  is  important  for  you  to  get  your  own  data  and  do  you  own 
analysis  in  parallel.  This  is  NOT  a  question  of  trust,  this 
is  prudence  and  good  management.  It  is  not  necessary  to 
find  a  problem,  just  an  inconsistency.  The  inconsistency 
highlights  an  area  where  questions  should  be  asked. 

TESTING  AND  TRAINING 


Testing  as  early  in  development  as  possible.  Yes,  this 
is  necessary,  but  in  the  proper  configuration.  The  item 
being  tested  must  be  representative  of  the  final  product,  or 
else  you  can  get  a  false  sense  of  security. 

Both  structured  and  unstructured  testing  conducted. 

Very  good.  An  added  benefit  is  user  involvement.  The  Air 
Force  uses  Independent  Verification  and  Validation  (IV&V) 
testing,  which  may  be  similar  to  the  unstructured  testing 
used  in  Host.  IV&V  testing  is  performed  by  an  independent 
contractor  who  makes  sure  that  the  item  doesn’t  do 
'unintended  things.*  Structured  testing  on  the  other  hand 
checks  that  the  item  does  what  it  is  intended  to  do.  IV&V 
depth  depends  on  the  ramifications  of  unintended 
occurrences . 

User  involved  in  planning  and  performing  test 
scenarios .  Agree . 

Site  personnel  trained  before  system  delivery.  Yes , 
this  is  necessary  but  it  is  not  often  done.  Not  sure  why  it 
is  not  often  done,  maybe  because  the  PM  hasn’t  planned  for 
it,  or  maybe  because  the  primary  focus  of  the  SPO  is  on 
'making  it  work*  versus  logistics  issues. 

After  delivery,  detailed  site  specific  testing 
conducted . 

CONCLUSION 


Fund  stability  is  also  essential.  Once  funding 
stability  is  gone,  you  really  have  program  problems.  A  new 
step  is  being  taken  at  AFSC  to  terminate  programs  if  there 
are  insufficient  RDT&E  funds  instead  of  stretching  the 
programs  out.  The  old  technique  of  stretching  the  program 
escalated  both  cost  and  schedule. 

The  program  management  'menu  for  success*  according  to 
Colonel  Tourino: 

1.  Schedule  -  realistic 

2.  Funding  -  stable 

3.  Communication  -  to  agree  on  what  you’re  doing 

4.  Tracking  -  progress  and  action  items 


i 
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General  Randolph’s  primary  goal  is  user  involvement. 

It  is  not  the  Job  of  AFSC  to  tell  the  user  "no",  but  rather 
to  tell  the  user  what  the  alternatives  and  consequences  are. 

There  has  been  a  revolution  at  AFSC  in  the  last  year. 
There  is  a  major  push  for  realistic  schedules.  HQ  AFSC  is 
telling  the  customer  that  this  much  time  and  money  is  needed 
to  meet  each  alternative.  It  is  up  to  the  user  to  select 
the  alternative,  but  not  dictate  the  schedule  and  cost.  HQ 
AFSC  is  becoming  more  proactive. 

There  is  no  magic  to  the  program  management  business, 
Just  coounon  sense.  It  is  necessary  to  take  a  stronger 
stand,  to  worry  about  something  other  than  CYA. 
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develop  an  acquisition  management  strategy  applicable  to  DOD 
program  management  that  would  help  program  managers  achieve 
acquisition  success.  A  hypothesized  management  strategy  was 
formulated  from  the  exploration  of  the  successful  Federal 
Aviation  Administration's  Host  computer  program.  This 
exploration  used  personal  interviews  to  identify  those 
management  elements  and  organizational  procedures  perceived 
by  the  28  respondents  to  contribute  to  Host  success.  The 
hypothesized  management  strategy  was  subsequently  evaluated 
by  experts  in  DOD  acquisition.  Through  personal  interviews 
each  element  in  the  management  strategy  was  evaluated  for 
necessity  to  achieve  program  success  and  applicability  to 
DOD  programs. 

The  conclusions  and  recommendations  of  the  study  were 
based  on  the  results  of  the  DOD  acquisition  expert  opinion 
survey  of  the  hypothesized  management  strategy.  The  result 
was  a  management  strategy  to  guide  DOD  program  managers  in 
achieving  acquisition  success  that  can  be  tailored  to  all 
programs.  ; 
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